{"paper_id":"2509.12764","title":"Myopic Optimality: why reinforcement learning portfolio management strategies lose money","authors":["Yuming Ma"],"paper_date":"2025-09-16","paper_type":"theoretical","paper_url":"https://arxiv.org/pdf/2509.12764v1","page_url":"https://thequant.space/flowcharts/myopic-optimality-why-reinforcement-learning-portfolio-mana/","scores":{"empirical_rigor":4,"math_complexity":9.5,"quadrant":"Lab Rats"},"proposes":{"status":"available","statement":"The paper proposes that myopic optimization (MO), implemented as a sequence of one‑period convex programs with execution/liquidation frictions, is sufficient for dynamic portfolio management and unifies policy‑gradient reinforcement learning (RL) via Malliavin calculus to derive risk shadow prices a","quote":"Myopic optimization (MO) outperforms reinforcement learning (RL) in portfolio management: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk. This gives dual gap and convergence results: geometric MO vs. RL floors.","summary":"Myopic optimization (MO) outperforms reinforcement learning (RL) in portfolio management: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk. We model execution/liquidation frictions with mark-to-market accounting. Using…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Market dynamics are action‑independent (AIM) and follow the specified stochastic processes.","type":"market","stated":"explicit","quote":"1 Introduction In this article we consider dynamic portfolio optimization for the buyside and the sellside mainly focusing on action-independent market (AIM) dynamics and standard execution/liquidation frictions with cases in control-affects-dynamics (CAD) market condition attached. In this regime, we show that myopic op­ timization (MO)-a sequence of one-period convex programs solved online-is dynamically","support":"full"},{"assumption":"Execution and liquidation frictions are modeled with mark‑to‑market accounting.","type":"execution","stated":"explicit","quote":"We model execution/liquidation frictions with mark-to-market accounting.","support":"full"},{"assumption":"Proportional trading costs are positive ( > 0 ).","type":"execution","stated":"explicit","quote":"With proportional trading costs >. > 0, optimal policies exhibit no-trade wedges and partial adjustment; small-cost band width 5 ex >.½, welfare-loss ex >.i [25, 43, 44, 26, 82].","support":"full"},{"assumption":"Control‑affects‑dynamics (CAD) market condition may be attached to the regime.","type":"model","stated":"explicit","quote":"1 Introduction In this article we consider dynamic portfolio optimization for the buyside and the sellside mainly focusing on action-independent market (AIM) dynamics and standard execution/liquidation frictions with cases in control-affects-dynamics (CAD) market condition attached.","support":"full"},{"assumption":"The objective functions are convex (or locally smooth) so that myopic convex programs are well‑posed.","type":"model","stated":"explicit","quote":"Myopic optimization (MO) treats dynamic programs as a sequence of one­ period convex programs solved at each time t, whereas the RL objective embeds each action's influence on future states and conducts long-horizon credit assign­ ment via a Bellman recursion or cross-time gradient propagation. In frictionless settings, myopic optimality holds under isoelastic preferences [73, 60, 59].","support":"full"},{"assumption":"Convergence analysis relies on smoothness and Polyak‑Łojasiewicz (PL) curvature of the risk functional.","type":"statistical","stated":"explicit","quote":"Convergence statements rely on smoothness and PL-type curvature; ill­ conditioned or sharply nonconvex objectives weaken constants.","support":"full"}]},"datasets":{"status":"none_found","items":[],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"none"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R4a","candidates":{"front_matter":"n/a","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"Standard execution/liquidation frictions with proportional costs are incorporated via mark‑to‑market accounting.","card_quote":"We model execution/liquidation frictions with mark-to-market accounting. With proportional trading costs >. > 0, optimal policies exhibit no-trade wedges and partial adjustment; small-cost band width 5 ex >.½, welfare-loss ex >.i [25, 43, 44, 26, 82].","meaning":"the paper reports no trading or execution result; the cost the extractor found is a model parameter"},"results":{"status":"available","evaluation_type":"none","findings":[{"finding":"Myopic optimization outperforms reinforcement learning: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.","kind":"performance","quote":"Myopic optimization (MO) outperforms reinforcement learning (RL) in portfolio management: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.","support":"full"},{"finding":"Geometric convergence of MO is established, while RL exhibits variance floors that limit convergence.","kind":"theoretical","quote":"This gives dual gap and convergence results: geometric MO vs. RL floors. Viewed as op­ timization flows, myopic schemes contract geometrically under PL geometry, while policy-gradient RL carries non-vanishing variance floors that stall con­ vergence unless variance is aggressively reduced [69, 9].","support":"full"},{"finding":"A Malliavin policy‑gradient contamination analysis quantifies phantom profit in RL and defines a CAD premium that can be plausibly positive.","kind":"theoretical","quote":"We quantify phantom profit in RL via Malliavin policy-gradient contamination analysis and define a control-affects-dynamics (CAD) premium of RL indicating plausibly positive.","support":"full"},{"finding":"Empirically, in most liquid AIM‑like environments RL adds at most 0‑1 bps per decision day; in rare CAD‑dominated niches it may reach low single‑digit bps per day.","kind":"performance","quote":"Nevertheless, empirically, in most liq­ uid AIM-like environments this nets out to 0-1 bps per decision day; in rare, 33 strongly CAD-dominated niches it can reach low single-digit bps/day, but those cases require carefully validated state feedback and tight leakage controls in RL solvers.","support":"full"},{"finding":"When market dynamics are AIM and portfolios are valued by prudent MtM convention, treating taker and maker execution as convex programs makes MO sufficient in practice.","kind":"theoretical","quote":"3 Conclusion and Discussion Across both taker and maker settings, when market dynamics are AIM and port­ folios are valuated by prudent MtM convention with spreads, impact, funding, and block discounts, by treating taker and maker execution as a sequence of con­ vex programs solved on decision ticks, MO is sufficient in practice.","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"none","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[{"paper":{"paper_id":"2306.10950","title":"Benchmarking Robustness of Deep Reinforcement Learning approaches to Online Portfolio Management","paper_date":"2023-06-19","authors":["Marc Velay","Bich-Liên Doan","Arpad Rimmel","Fabrice Popineau","Fabrice Daniel"],"empirical_rigor":6,"page_url":"https://thequant.space/flowcharts/benchmarking-robustness-of-deep-reinforcement-learning-appro/"},"question":"Do reinforcement learning portfolio management strategies provide robust and profitable performance?","this_paper_finding":"Myopic optimization outperforms reinforcement learning: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.","other_paper_finding":"Most deep reinforcement learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.","other_paper_quote":"We found that most Deep Reinforcement Learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.","note":"Both findings agree that RL methods perform poorly: A reports lack of robustness and quick degradation, while B shows lower/negative returns, higher variance, costs, CVaR, and model risk.","confidence":0.912}],"contradicts":[],"qualifies":[],"compared_with":25,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T19:47:50Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}