Paper: arXiv 2512.10913

Authors: Mohammad Rezoanul Hoque, Md Meftahul Ferdaus, M. Kabir Hassan

Abstract

Reinforcement learning (RL) is an innovative approach to financial decision making, offering specialized solutions to complex investment problems where traditional methods fail. This review analyzes 167 articles from 2017–2025, focusing on market making, portfolio optimization, and algorithmic trading. It identifies key performance issues and challenges in RL for finance. Generally, RL offers advantages over traditional methods, particularly in market making. This study proposes a unified framework to address common concerns such as explainability, robustness, and deployment feasibility. Empirical evidence with synthetic data suggests that implementation quality and domain knowledge often outweigh algorithmic complexity. The study highlights the need for interpretable RL architectures for regulatory compliance, enhanced robustness in nonstationary environments, and standardized benchmarking protocols. Organizations should focus less on algorithm sophistication and more on market microstructure, regulatory constraints, and risk management in decision-making.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper reviews complex RL algorithms and theoretical finance concepts, indicating substantial mathematical density, but its empirical rigor is limited as it analyzes 167 articles using synthetic data rather than presenting new backtests or real-world datasets.

Research Flowchart

  flowchart TD
  A["Research Goal: Review RL in Finance<br>(2017-2025, n=167)"] --> B{"Systematic Literature Review"}
  B --> C["Topics Analyzed<br>Market Making | Portfolio Optimization | Algorithmic Trading"]
  C --> D["Computational Synthesis<br>Performance Metrics vs. Challenges"]
  D --> E["Key Findings & Outcomes"]
  E --> F["Framework & Recommendations<br>Focus: Microstructure & Risk over Complexity"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The study proposes a unified framework to address common concerns such as explainability, robustness, and deployment feasibility in reinforcement learning for finance.

Source sentence
This study proposes a unified framework to address common concerns such as explainability, robustness, and deployment feasibility.

Assumes 4

  • The synthetic dataset reproduces key statistical patterns observed in the literature and preserves relationships among variables. data
    Source sentence
    Empirical Validation Framework To validate the meta-analysis findings, a synthetic dataset was developed that repro-duces key statistical patterns observed in the literature. The data generation process preserves the statistical relationships observed in the literature while enabling controlled analysis of performance drivers. 4.
  • Transaction costs and market impact can be captured through explicit reward‑structure constraints in the RL formulation. execution
    Source sentence
    Recent portfolio op-timization research focuses on incorporating realistic limitations, such as transaction costs, into RL frameworks. The RL approach enables objective optimization by con-sidering transaction costs and market impact constraints through explicit reward structure constraints, along with legal and regulatory constraints using effectively designed reward functions.
  • Continuous action spaces are appropriate for portfolio weight and market‑making decisions, enabling RL methods to adapt to market microstructure. model
    Source sentence
    Due to DDPG’s capability with continuous action spaces for portfolio weights, it is utilized in optimizing portfolio management. This improvement likely results from the adaptability and continuous nature of RL methods, which better capture the dy-namic market microstructure and efficiently solve complex multi-objective functions that are difficult for traditional market making methods.
  • Performance gains reported for market making and cryptocurrency trading reflect typical ranges across multiple implementations rather than single‑study results. statistical
    Source sentence
    The performance metrics represent aggregated results from the meta-analysis of 167 studies, with Sharpe ratios reflecting typical performance ranges observed across mul-tiple implementations within each algorithm-application category rather than results from individual studies. Market making applications typically exhibit the highest performance gains, as demon-strated in Figure 5 where market making shows the

Data

  • synthetic dataset synthetic unknown
    Source sentence
    Empirical Validation Framework To validate the meta-analysis findings, a synthetic dataset was developed that repro-duces key statistical patterns observed in the literature. The synthetic dataset includes 167 studies with variables representing feature dimensions, number of as-sets, training periods, algorithm types, application domains, and performance metrics.

Costs

Not applicable The paper reports no trading or execution result; the cost the extractor found is a model parameter.

The paper-text extraction found a cost parameter inside a model (Transaction costs are incorporated via explicit reward‑structure constraints, including cost and market‑impact considerations.); the classifier found no reported trading result, so costs are not applicable.

Finds 4

  • Market making applications exhibit the highest performance gains among RL‑enabled financial tasks, as shown by meta‑analysis of 167 studies. performance figure from elsewhere in the paper
    Source sentence
    Market making applications typically exhibit the highest performance gains, as demon-strated in Figure 5 where market making shows the highest RL premium among all appli-cation domains.
  • Synthetic validation indicates that implementation quality and domain expertise outweigh algorithmic complexity for successful RL in finance. performance
    Source sentence
    Empirical evidence with synthetic data suggests that implementation quality and domain knowledge often outweigh algorithmic complexity. The meta-analysis indicates that successful RL in finance relies more on quality implementation, domain expertise, and data quality than complex algorithms.
  • RL methods outperform traditional approaches in market making by adapting to continuous control problems and optimizing bid‑ask spread capture. performance
    Source sentence
    Market making involves managing inventory risk to maintain security levels within a target range while profiting from the bid-ask spread. In contrast, RL methods adapt to market conditions and manage multiple objectives like bid-ask spread capture, inventory, and risk.
  • Cryptocurrency trading shows the second‑highest performance, driven by high volatility and market inefficiencies that RL can exploit. performance
    Source sentence
    Cryptocurrency trading applications follow as the second-highest performing domain, likely due to the greater re-active volatility and inefficiencies in these markets, which can be effectively exploited with RL algorithms.

Checked by others

No other paper in the archive has been found to support or contradict these findings. Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (124)

  1. A Review of Reinforcement Learning in Financial Markets (2024)
  2. How are reinforcement learning and deep learning algorithms used for big data based decision making in financial industries–A review and research agenda (2022)
  3. Cryptocurrency trading strategy based on deep reinforcement learning (2024)
  4. Reinforcement learning for market making: A theoretical framework — → 2410.14504
  5. Deep reinforcement learning for portfolio optimization with ESG constraints (2024)
  6. Risk-aware reinforcement learning for financial decision making (2023)
  7. Multi-agent reinforcement learning for competitive trading (2023)
  8. Optimal execution with reinforcement learning under market impact (2024)
and 116 more
  1. Explainable reinforcement learning for financial applications (2023)
  2. Regulatory challenges for AI in financial markets (2024)
  3. Safe exploration in financial reinforcement learning (2023)
  4. Quantum reinforcement learning for portfolio optimization (2024)
  5. ESG-aware reinforcement learning for sustainable investing (2023)
  6. Central bank digital currencies and reinforcement learning (2024)
  7. Blockchain-based reinforcement learning for decentralized finance (2023)
  8. Edge computing for high-frequency trading with reinforcement learning (2024)
  9. Validation frameworks for reinforcement learning in finance (2023)
  10. Standardization needs for financial AI systems (2024)
  11. International coordination of AI regulation in finance (2023)
  12. Industry best practices for reinforcement learning deployment (2024)
  13. Machine Learning (1997)
  14. Foundations of Machine Learning (2018)
  15. Reinforcement Learning: An Introduction (2018)
  16. Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case (1969)
  17. Portfolio Selection (1952)
  18. Options, Futures, and Other Derivatives (2017)
  19. The Pricing of Options and Corporate Liabilities (1973)
  20. Analysis of Financial Time Series (2010)
  21. Dynamic Programming (1957)
  22. Dynamic Programming and Optimal Control (2012)
  23. Q-learning (1992)
  24. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning (1992)
  25. Actor-Critic Algorithms (2000)
  26. Human-level Control through Deep Reinforcement Learning (2015)
  27. Proximal Policy Optimization Algorithms (2017) — link
  28. Continuous Control with Deep Reinforcement Learning (2015) — link
  29. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018)
  30. A Comprehensive Survey on Safe Reinforcement Learning (2015)
  31. Challenges of Real-World Reinforcement Learning (2019) — link
  32. Deep Learning (2016)
  33. Learning to trade via direct reinforcement (2001)
  34. Forecasting stock prices from the limit order book using convolutional neural networks (2017)
  35. A review of the past and future of electronic trading and its impact on market structure (2013)
  36. Towards automated trading based on reinforcement learning (2018)
  37. Deep learning for finance: deep portfolios (2017)
  38. … and the cross-section of expected returns (2016)
  39. Empirical properties of asset returns: stylized facts and statistical issues (2001)
  40. An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown (2017)
  41. Towards a rigorous science of interpretable machine learning (2017) — link
  42. High-frequency trading (2017)
  43. Explainable reinforcement learning: a survey (2020)
  44. Algorithmic and high-frequency trading (2015)
  45. Optimal execution of portfolio transactions (2001)
  46. Market making via reinforcement learning (2018)
  47. Quantitative risk management: concepts, techniques and tools-revised edition (2015)
  48. Modern perspectives on reinforcement learning in finance (2019)
  49. Machine learning for risk management (2020)
  50. The probability of backtest overfitting (2014)
  51. Advances in financial machine learning (2018)
  52. High-frequency trading: a practical guide to algorithmic strategies and trading systems (2013)
  53. Quantitative equity investing: techniques and strategies (2010)
  54. Analysis of financial time series (2005)
  55. Survivorship bias in performance studies (1992)
  56. A sparse model of demand for news: Evidence from the options market (2019)
  57. The black swan: The impact of the highly improbable (2007)
  58. A theory of learning from different domains (2010)
  59. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  60. Dueling network architectures for deep reinforcement learning (2016)
  61. Rainbow: Combining improvements in deep reinforcement learning (2018)
  62. A distributional perspective on reinforcement learning (2017)
  63. Trust region policy optimization (2015)
  64. Asynchronous methods for deep reinforcement learning (2016)
  65. Addressing function approximation error in actor-critic methods (2018)
  66. Deep reinforcement learning in a handful of trials using probabilistic dynamics models (2018)
  67. Neural network dynamics for model-based control (2018)
  68. Dyna, an integrated architecture for learning, planning, and reacting (1991)
  69. Multi-agent actor-critic for mixed cooperative-competitive environments (2017)
  70. QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning (2018)
  71. Counterfactual multi-agent policy gradients (2018)
  72. Learning multi-level hierarchies with hindsight (2019)
  73. FeUdal networks for hierarchical reinforcement learning (2017)
  74. The option-critic architecture (2017)
  75. Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading (2020)
  76. Deep learning for portfolio optimization (2020)
  77. Reinforcement learning in economics and finance (2021)
  78. Deep robust reinforcement learning for practical algorithmic trading (2019)
  79. Bridging the gap between Markowitz planning and deep reinforcement learning (2021)
  80. Deep reinforcement learning for multi-period portfolio optimization (2021)
  81. An application of deep reinforcement learning to algorithmic trading (2021)
  82. CEFi vs. DeFi–comparing centralized to decentralized finance (2021) — link
  83. QLBS: Q-learner in the Black-Scholes (-Merton) worlds (2020)
  84. Reinforcement learning applied to Forex trading (2018)
  85. Modelling stock markets by multi-agent reinforcement learning (2021)
  86. A multi-agent and machine learning based online adaptive energy management system for plug-in hybrid electric vehicles (2020)
  87. Multi-agent deep reinforcement learning for portfolio management (2021)
  88. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  89. Recent advances in reinforcement learning in finance (2023)
  90. Machine learning for trading (2017)
  91. Deep robust reinforcement learning for practical algorithmic trading (2019)
  92. Deep hedging (2019)
  93. Modelling stock markets by multi-agent reinforcement learning (2021)
  94. Recent advances in reinforcement learning in finance (2023)
  95. Reinforcement learning applied to Forex trading (2018)
  96. Using deep learning to detect price change indications in financial markets (2017)
  97. Network structure and systemic risk in banking systems (2010)
  98. Handbook of financial risk management (2020)
  99. Digital finance and FinTech: current research and future research directions (2017)
  100. Deep learning with long short-term memory networks for financial market predictions (2018)
  101. Financial time series forecasting with deep learning: A systematic literature review: 2005–2019 (2020)
  102. Stock price prediction via discovering multi-frequency trading patterns (2017)
  103. Deep transformer models for time series forecasting: The influenza prevalence case (2020) — link
  104. Time-series generative adversarial networks (2019)
  105. Temporal relational ranking for stock prediction (2019)
  106. Learning to reinforcement learn (2016) — link
  107. Empirical market microstructure: The institutions, economics, and econometrics of securities trading (2007)
  108. Optimal control of execution costs (1998)
  109. Value at risk: the new benchmark for managing financial risk (2007)
  110. Optimal execution with nonlinear impact functions and trading-enhanced risk (2003)
  111. Limit order books (2013)
  112. The implementation shortfall: Paper versus reality (1988)
  113. Asset Pricing and Machine Learning: A critical review (2022) — link
  114. Behavioral finance: Evolution from the classical theory and remarks (2024) — link
  115. The evolution of stock market efficiency over time: A survey of the empirical literature (2011) — link
  116. From Econometrics to Machine Learning: Transforming Empirical Asset Pricing (2025)

1 of 124 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.