Paper: arXiv 2406.08013

Authors: Sven Goluža, Tomislav Kovačević, Tessa Bauman, Zvonko Kostanjčar

Abstract

Deep reinforcement learning (DRL) is a well-suited approach to financial decision-making, where an agent makes decisions based on its trading strategy developed from market observations. Existing DRL intraday trading strategies mainly use price-based features to construct the state space. They neglect the contextual information related to the position of the strategy, which is an important aspect given the sequential nature of intraday trading. In this study, we propose a novel DRL model for intraday trading that introduces positional features encapsulating the contextual information into its sparse state space. The model is evaluated over an extended period of almost a decade and across various assets including commodities and foreign exchange securities, taking transaction costs into account. The results show a notable performance in terms of profitability and risk-adjusted metrics. The feature importance results show that each feature incorporating contextual information contributes to the overall performance of the model. Additionally, through an exploration of the agent’s intraday trading activity, we unveil patterns that substantiate the effectiveness of our proposed model.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced deep reinforcement learning techniques with specific architectural choices (like sparse state spaces) and discusses sequential decision-making theory, indicating high mathematical complexity. It demonstrates high empirical rigor by evaluating the model over a decade across multiple asset classes, accounting for transaction costs, and performing feature importance analysis.

Research Flowchart

  flowchart TD
  A["Research Goal: Enhance DRL for Intraday Trading"] --> B["Methodology: DRL Model with Positional State"]
  B --> C["Data: Multi-Asset History<br>with Transaction Costs"]
  C --> D["Process: Sparse State Space Construction<br>(Price + Position Context)"]
  D --> E["Process: Deep Reinforcement Learning<br>Agent Training & Decision Making"]
  E --> F["Key Findings: High Profitability &<br>Risk-Adjusted Returns"]
  F --> G["Outcome: Validated Positional<br>Feature Importance & Trading Patterns"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a novel deep reinforcement learning model for intraday trading that incorporates positional features into a sparse state space.

Source sentence
This paper introduces three contributions. Primarily, we present a novel DRL model for intraday trading strategies, which integrates positional features as a pivotal component.

Assumes 5

  • Transaction costs consist of commission, slippage, and negligible market impact. execution
    Source sentence
    The transaction costs txt include commission, slippage, and market impact. Commission cost estimates (COM) are expressed in basis points (BP)
  • Commission is measured in basis points (1BP = 0.01%). model
    Source sentence
    Commission cost estimates (COM) are expressed in basis points (BP), where 1BP equals 0.01%.
  • Trades are executed at the next period's open price for slippage estimation. execution
    Source sentence
    Slippage is conservatively approximated by assuming that trades execute at the Open price pO t+1 of the next time step
  • The action space represents the position to hold, not direct buy/sell orders. model
    Source sentence
    This action space also represents the position the agent needs to take in the asset, not the actual trading decisions of buying or selling the asset
Show 1 more
  • Reward is additive logarithmic return at each time step. statistical
    Source sentence
    To satisfy the additive rewards required by the DRL optimization goal, we define the reward function as the logarithmic return at each time step

Data

Backtest sample: 2013 – 2022 · commodities and foreign exchange · multi-asset · intraday

  • Private market data · private data vendor market-prices commercial
    nearly a decade · commodities and foreign exchange securities
    Source sentence
    It is evaluated across various assets, including commodities and foreign exchange securities, over an extensive evaluation period of nearly a decade.
  • Data from FirstRateData · FirstRateData market-prices commercial
    Source sentence
    Data is provided by a private data vendor at https://firstratedata.com.

Costs

Modeled Costs are part of the reported headline results.

Commission expressed in basis points, slippage assumed at next open price, market impact considered negligible for one-unit trades.

Source sentence
The transaction costs txt include commission, slippage, and market impact. Commission cost estimates (COM) are expressed in basis points (BP)

Finds 3

Reported performance (Deep reinforcement learning (DRL) intraday trading model): Sharpe 3.81 (benchmark -0.16)

  • The DRL agent outperforms benchmark strategies in most assets, delivering high Sharpe ratios after accounting for realistic transaction costs. performance
    Source sentence
    Our DRL agent demonstrates notable performance, outperforming the benchmarks in most assets and yielding high Sharpe ratios after accounting for realistic transaction costs.
  • Models with positional features achieve higher Sharpe ratios than those without, especially under higher commission fees. performance
    Source sentence
    Our findings indicate that, on average, the model with positional features outperforms the alternative across higher commission fees.
  • The agent concentrates trades early in the day and reduces activity later, suggesting learned avoidance of higher volatility near market close. other
    Source sentence
    The agent initiates the majority of trades early in the day, gradually reducing its trading activity as the day progresses toward its conclusion.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (51)

  1. The index of general nonlinear D A E S (1995)
  2. Clinical implications of dysregulated cytokine production (2000) — link
  3. Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations (1995)
  4. Algorithms for C omputer A lgebra (1992)
  5. Conductive P olymers (1981)
  6. Isolation and characterization of plasmid deoxyribonucleic acid from Streptomyces fradiae (1978)
  7. Quantum scissors: teleportation of single-mode optical states by means of a nonlocal single photon (2002)
  8. Mixing induced CP asymmetries in inclusive B decays (1997) — link
and 43 more
  1. deep SIP : deep learning of S upernova I a P arameters (2020)
  2. Dark Energy Survey Year 1 Results: Constraints on Extended Cosmological Models from Galaxy Clustering and Weak Lensing (2019) — link
  3. Reinforcement learning: An introduction (2018)
  4. Inside the black box: A simple guide to quantitative and high frequency trading (2013)
  5. Learning dexterous in-hand manipulation (2020)
  6. Mastering the game of go without human knowledge (2017)
  7. Human-level control through deep reinforcement learning (2015)
  8. Deep reinforcement learning for autonomous driving: A survey (2021)
  9. Deep learning (2016)
  10. Efficient capital markets: A review of theory and empirical work (1970)
  11. Time series momentum (2012) — link
  12. Cross-sectional and time-series determinants of momentum returns (2002)
  13. Mean reversion in stock prices: Evidence and implications (1988) — link
  14. Intraday market predictability: A machine learning approach (2023) — link
  15. Intraday Stock Predictability Everywhere (2023)
  16. Intraday return predictability in the cryptocurrency markets: Momentum, reversal, or both (2022) — link
  17. Intraday patterns in the cross-section of stock returns (2010)
  18. Deep direct reinforcement learning for financial signal representation and trading (2016) — link
  19. Learning to trade via direct reinforcement (2001) — link
  20. Financial trading as a game: A deep reinforcement learning approach (2018)
  21. Effect of labeling algorithms on financial performance metrics (2022)
  22. Universal features of price formation in financial markets: perspectives from deep learning (2019) — link
  23. DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
  24. A multi-objective deep reinforcement learning approach for stock index future’s intraday trading (2017)
  25. Deep robust reinforcement learning for practical algorithmic trading (2019) — link
  26. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  27. Imitation Learning for Financial Applications (2023)
  28. Proximal policy optimization algorithms (2017)
  29. Deep reinforcement learning for trading (2020) — link
  30. Enhancing time-series momentum strategies using deep neural networks (2019) — link
  31. Statistical modeling of high-frequency financial data (2011) — link
  32. Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
  33. High-dimensional continuous control using generalized advantage estimation (2015)
  34. Continuous futures contracts methodology for backtesting (2020)
  35. Pseudomathematics and financial charlatanism: The effects of backtest over fitting on out-of-sample performance (2014) — link
  36. Deep order flow imbalance: Extracting alpha at multiple horizons from the limit order book (2023) — link
  37. Adam: A method for stochastic optimization (2014)
  38. Technical analysis of the financial markets: A comprehensive guide to trading methods and applications (1999)
  39. Does algorithmic trading improve liquidity? (2011) — link
  40. Performance of the Average Directional Index as a market timing tool for the most actively traded USD based currency pairs (2018) — link
  41. Forecasting East Asian indices futures via a novel hybrid of wavelet-PCA denoising and artificial neural network models (2016) — link
  42. Bitcoin transaction strategy construction based on deep reinforcement learning (2021) — link
  43. Evolving deep gated recurrent unit using improved marine predator algorithm for profit prediction based on financial accounting information system (2023) — link

0 of 51 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.