{"paper_id":"2511.00190","title":"Deep reinforcement learning for optimal trading with partial information","authors":["Andrea Macrì","Sebastian Jaimungal","Fabrizio Lillo"],"paper_date":"2025-10-31","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2511.00190v1","page_url":"https://thequant.space/flowcharts/deep-reinforcement-learning-for-optimal-trading-with-partial/","scores":{"empirical_rigor":6,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes three Deep Deterministic Policy Gradient (DDPG) algorithms that integrate Gated Recurrent Unit (GRU) networks to solve an optimal trading problem with a mean‑reverting Ornstein‑Uhlenbeck signal and latent regime information.","quote":"To address this problem, we propose three Deep Deterministic Policy Gradient (DDPG)–based algorithms that integrate Gated Recurrent Unit (GRU) networks to capture temporal dependencies in the signal.","summary":"Reinforcement Learning (RL) applied to financial problems has been the subject of a lively area of research. The use of RL for optimal trading strategies that exploit latent information in the market is, to the best of our knowledge, not widely tackled. In this paper we study an optimal trading…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The trading signal follows an Ornstein‑Uhlenbeck process with regime‑switching dynamics governed by independent Markov chains.","type":"model","stated":"explicit","quote":"we study an optimal trading problem, where a trading signal follows an Ornstein–Uhlenbeck process with regime‑switching dynamics."},{"assumption":"Transaction costs are a linear per‑unit fee λ ≥ 0 applied to each trade volume.","type":"execution","stated":"explicit","quote":"Whenever the trader performs a trade of volume q_t, she pays a transaction cost per unit volume of λ ≥0."},{"assumption":"The agent can hold inventory within a bounded interval [‑I_max, I_max] and starts with zero inventory.","type":"execution","stated":"explicit","quote":"the agent can hold inventories within the interval I ∈[‑Imax, Imax] and we always start the testing episodes of trading with S0 = 1 and I0 = 0."},{"assumption":"The trader is risk‑neutral and maximises the expected discounted sum of rewards.","type":"model","stated":"explicit","quote":"a risk neutral trader who maximises the expected discounted profits from trading."},{"assumption":"The latent parameters (mean reversion, speed, volatility) are filtered from observations of the signal.","type":"data","stated":"explicit","quote":"The latent parameters driving mean reversion, speed, and volatility are filtered from observations of the signal."},{"assumption":"The pair‑trading assets are co‑integrated, allowing a stationary linear combination to be used as the trading portfolio.","type":"model","stated":"explicit","quote":"We see that both tests indicate strong evidence of at least one co‑integrating vector"}]},"datasets":{"status":"available","items":[{"name":"Intel (INTC) and Merrill Lynch Semiconductor ETF (SMH) high‑frequency limit order book dat","provider":"LOBSTER","kind":"order-book","access":"commercial","period":"2025‑08‑29 to 2025‑09‑05","universe":"Two NASDAQ‑listed assets (INTC, SMH)","quote":"We consider two assets traded on the NASDAQ between the 29th of August 2025 and the 5th of September 2025, the Intel (INTC) stock and the Merril Lynch semi‑conductors ETF (SMH)","known_id":"lobster"}],"known":[{"id":"lobster","name":"LOBSTER","provider":"LOBSTER","kind":"order-book","access":"commercial","quote":"We use high frequency observations of the limit order book for both the assets, downloaded from the LOBSTER database, we consider data up to the first level of the order book and we select only trade events, so that each element considered in the time series represents a transaction.","mentions":1,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"intraday-bars"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":null,"detail":"A per‑unit transaction cost λ ≥ 0 is subtracted from the reward for each trade volume.","quote":"Whenever the trader performs a trade of volume q_t, she pays a transaction cost per unit volume of λ ≥0. As a result, the reward r_t from the trading action at time t is r_t(q_t, S_t, S_{t+1}, λ) := I_{t+1} (S_{t+1}‑S_t) ‑λ |q_t| .","source":"evidence_card","rule":"R2","candidates":{"front_matter":"unclear","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"A per‑unit transaction cost λ ≥ 0 is subtracted from the reward for each trade volume.","card_quote":"Whenever the trader performs a trade of volume q_t, she pays a transaction cost per unit volume of λ ≥0. As a result, the reward r_t from the trading action at time t is r_t(q_t, S_t, S_{t+1}, λ) := I_{t+1} (S_{t+1}‑S_t) ‑λ |q_t| .","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"prob‑DDPG consistently achieves the highest cumulative trading rewards and the most interpretable strategies in both synthetic and real‑world pair‑trading experiments.","kind":"performance","quote":"we find that prob‑DDPG achieves superior cumulative rewards and exhibits more interpretable strategies."},{"finding":"reg‑DDPG provides limited benefits, offering little improvement over baseline.","kind":"performance","quote":"By contrast, reg‑DDPG provides limited benefits."},{"finding":"hid‑DDPG yields intermediate performance with less interpretable strategies.","kind":"performance","quote":"hid‑DDPG offers intermediate performance with less interpretable strategies."},{"finding":"Providing posterior probability estimates of the mean‑reversion regimes leads to the highest trading rewards, confirming the importance of structured information.","kind":"performance","quote":"The numerical results show that the agent, when equipped with posterior probability estimates of the mean reversion regimes (which it learns in a first stage), achieves the highest trading rewards."},{"finding":"The conclusions about the superiority of prob‑DDPG hold when applied to real market pair‑trading data.","kind":"performance","quote":"Finally, we find that these conclusions remain valid when the agent faces trading problems with real market data, when dealing with a pair trading problem for two co‑integrated assets."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":27,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":1},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:23:08Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}