Paper: arXiv 2406.08013
Authors: Sven Goluža, Tomislav Kovačević, Tessa Bauman, Zvonko Kostanjčar
Abstract
Deep reinforcement learning (DRL) is a well-suited approach to financial decision-making, where an agent makes decisions based on its trading strategy developed from market observations. Existing DRL intraday trading strategies mainly use price-based features to construct the state space. They neglect the contextual information related to the position of the strategy, which is an important aspect given the sequential nature of intraday trading. In this study, we propose a novel DRL model for intraday trading that introduces positional features encapsulating the contextual information into its sparse state space. The model is evaluated over an extended period of almost a decade and across various assets including commodities and foreign exchange securities, taking transaction costs into account. The results show a notable performance in terms of profitability and risk-adjusted metrics. The feature importance results show that each feature incorporating contextual information contributes to the overall performance of the model. Additionally, through an exploration of the agent’s intraday trading activity, we unveil patterns that substantiate the effectiveness of our proposed model.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced deep reinforcement learning techniques with specific architectural choices (like sparse state spaces) and discusses sequential decision-making theory, indicating high mathematical complexity. It demonstrates high empirical rigor by evaluating the model over a decade across multiple asset classes, accounting for transaction costs, and performing feature importance analysis.
Research Flowchart
flowchart TD A["Research Goal: Enhance DRL for Intraday Trading"] --> B["Methodology: DRL Model with Positional State"] B --> C["Data: Multi-Asset History<br>with Transaction Costs"] C --> D["Process: Sparse State Space Construction<br>(Price + Position Context)"] D --> E["Process: Deep Reinforcement Learning<br>Agent Training & Decision Making"] E --> F["Key Findings: High Profitability &<br>Risk-Adjusted Returns"] F --> G["Outcome: Validated Positional<br>Feature Importance & Trading Patterns"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a novel deep reinforcement learning model for intraday trading that incorporates positional features into a sparse state space.
Source sentence
This paper introduces three contributions. Primarily, we present a novel DRL model for intraday trading strategies, which integrates positional features as a pivotal component.
Assumes 5
- Transaction costs consist of commission, slippage, and negligible market impact. execution
Source sentence
The transaction costs txt include commission, slippage, and market impact. Commission cost estimates (COM) are expressed in basis points (BP)
- Commission is measured in basis points (1BP = 0.01%). model
Source sentence
Commission cost estimates (COM) are expressed in basis points (BP), where 1BP equals 0.01%.
- Trades are executed at the next period's open price for slippage estimation. execution
Source sentence
Slippage is conservatively approximated by assuming that trades execute at the Open price pO t+1 of the next time step
- The action space represents the position to hold, not direct buy/sell orders. model
Source sentence
This action space also represents the position the agent needs to take in the asset, not the actual trading decisions of buying or selling the asset
Show 1 more
- Reward is additive logarithmic return at each time step. statistical
Source sentence
To satisfy the additive rewards required by the DRL optimization goal, we define the reward function as the logarithmic return at each time step
Data
2013 – 2022 · commodities and foreign exchange · multi-asset · intraday
- Private market data market-prices commercial
Source sentence
It is evaluated across various assets, including commodities and foreign exchange securities, over an extensive evaluation period of nearly a decade.
- Data from FirstRateData market-prices commercial
Source sentence
Data is provided by a private data vendor at https://firstratedata.com.
Costs
Modeled
Commission expressed in basis points, slippage assumed at next open price, market impact considered negligible for one-unit trades.
Source sentence
The transaction costs txt include commission, slippage, and market impact. Commission cost estimates (COM) are expressed in basis points (BP)
Finds 3
Sharpe 3.81 (benchmark -0.16)
- The DRL agent outperforms benchmark strategies in most assets, delivering high Sharpe ratios after accounting for realistic transaction costs. performance
Source sentence
Our DRL agent demonstrates notable performance, outperforming the benchmarks in most assets and yielding high Sharpe ratios after accounting for realistic transaction costs.
- Models with positional features achieve higher Sharpe ratios than those without, especially under higher commission fees. performance
Source sentence
Our findings indicate that, on average, the model with positional features outperforms the alternative across higher commission fees.
- The agent concentrates trades early in the day and reduces activity later, suggesting learned avoidance of higher volatility near market close. other
Source sentence
The agent initiates the majority of trades early in the day, gradually reducing its trading activity as the day progresses toward its conclusion.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (51)
- The index of general nonlinear D A E S (1995)
- Clinical implications of dysregulated cytokine production (2000) — link
- Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations (1995)
- Algorithms for C omputer A lgebra (1992)
- Conductive P olymers (1981)
- Isolation and characterization of plasmid deoxyribonucleic acid from Streptomyces fradiae (1978)
- Quantum scissors: teleportation of single-mode optical states by means of a nonlocal single photon (2002)
- Mixing induced CP asymmetries in inclusive B decays (1997) — link
and 43 more
- deep SIP : deep learning of S upernova I a P arameters (2020)
- Dark Energy Survey Year 1 Results: Constraints on Extended Cosmological Models from Galaxy Clustering and Weak Lensing (2019) — link
- Reinforcement learning: An introduction (2018)
- Inside the black box: A simple guide to quantitative and high frequency trading (2013)
- Learning dexterous in-hand manipulation (2020)
- Mastering the game of go without human knowledge (2017)
- Human-level control through deep reinforcement learning (2015)
- Deep reinforcement learning for autonomous driving: A survey (2021)
- Deep learning (2016)
- Efficient capital markets: A review of theory and empirical work (1970)
- Time series momentum (2012) — link
- Cross-sectional and time-series determinants of momentum returns (2002)
- Mean reversion in stock prices: Evidence and implications (1988) — link
- Intraday market predictability: A machine learning approach (2023) — link
- Intraday Stock Predictability Everywhere (2023)
- Intraday return predictability in the cryptocurrency markets: Momentum, reversal, or both (2022) — link
- Intraday patterns in the cross-section of stock returns (2010)
- Deep direct reinforcement learning for financial signal representation and trading (2016) — link
- Learning to trade via direct reinforcement (2001) — link
- Financial trading as a game: A deep reinforcement learning approach (2018)
- Effect of labeling algorithms on financial performance metrics (2022)
- Universal features of price formation in financial markets: perspectives from deep learning (2019) — link
- DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
- A multi-objective deep reinforcement learning approach for stock index future’s intraday trading (2017)
- Deep robust reinforcement learning for practical algorithmic trading (2019) — link
- Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
- Imitation Learning for Financial Applications (2023)
- Proximal policy optimization algorithms (2017)
- Deep reinforcement learning for trading (2020) — link
- Enhancing time-series momentum strategies using deep neural networks (2019) — link
- Statistical modeling of high-frequency financial data (2011) — link
- Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
- High-dimensional continuous control using generalized advantage estimation (2015)
- Continuous futures contracts methodology for backtesting (2020)
- Pseudomathematics and financial charlatanism: The effects of backtest over fitting on out-of-sample performance (2014) — link
- Deep order flow imbalance: Extracting alpha at multiple horizons from the limit order book (2023) — link
- Adam: A method for stochastic optimization (2014)
- Technical analysis of the financial markets: A comprehensive guide to trading methods and applications (1999)
- Does algorithmic trading improve liquidity? (2011) — link
- Performance of the Average Directional Index as a market timing tool for the most actively traded USD based currency pairs (2018) — link
- Forecasting East Asian indices futures via a novel hybrid of wavelet-PCA denoising and artificial neural network models (2016) — link
- Bitcoin transaction strategy construction based on deep reinforcement learning (2021) — link
- Evolving deep gated recurrent unit using improved marine predator algorithm for profit prediction based on financial accounting information system (2023) — link
0 of 51 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.