Paper: arXiv 2309.00630
Authors: Jonas Hanetho
Abstract
Algorithmic trading has gained attention due to its potential for generating superior returns. This paper investigates the effectiveness of deep reinforcement learning (DRL) methods in algorithmic commodities trading. It formulates the commodities trading problem as a continuous, discrete-time stochastic dynamical system. The proposed system employs a novel time-discretization scheme that adapts to market volatility, enhancing the statistical properties of subsampled financial time series. To optimize transaction-cost- and risk-sensitive trading agents, two policy gradient algorithms, namely actor-based and actor-critic-based approaches, are introduced. These agents utilize CNNs and LSTMs as parametric function approximators to map historical price observations to market positions.Backtesting on front-month natural gas futures demonstrates that DRL models increase the Sharpe ratio by $83%$ compared to the buy-and-hold baseline. Additionally, the risk profile of the agents can be customized through a hyperparameter that regulates risk sensitivity in the reward function during the optimization process. The actor-based models outperform the actor-critic-based models, while the CNN-based models show a slight performance advantage over the LSTM-based models.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 6.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper uses advanced stochastic calculus and deep RL theory for continuous action spaces and policy gradients, while providing backtested results on natural gas futures with specific performance metrics.
Research Flowchart
flowchart TD A["<b>Research Goal</b><br>Effectiveness of DRL in<br>Commodities Trading?"] --> B["<b>Methodology</b><br>Novel Volatility-Adaptive<br>Time-Discretization"] B --> C["<b>Agent Optimization</b><br>Policy Gradient Algorithms<br>Actor vs. Actor-Critic"] C --> D["<b>Computational Process</b><br>CNN & LSTM Function<br>Approximators"] D --> E["<b>Implementation</b><br>Backtesting on<br>Front-Month Natural Gas Futures"] E --> F["<b>Key Findings</b><br>83% ↑ Sharpe Ratio<br>Actor > Actor-Critic<br>Customizable Risk Profile"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a deep reinforcement learning framework for commodities trading that includes a novel time‑discretization scheme adaptive to market volatility and two policy‑gradient algorithms (actor‑based and actor‑critic‑based) using CNN and LSTM networks.
Source sentence
It formulates the commodities trading problem as a continuous, discrete-time stochastic dynamical system. The proposed system employs a novel time-discretization scheme that adapts to market volatility, enhancing the statistical properties of subsampled financial time series.
Assumes 2
- No slippage; all trades can be executed at the exact quoted price. execution
Source sentence
1. No slippage; all trades can be executed at the exact quoted price.
- No market impact; the capital invested by the agent is not significant enough to move the market. market
Source sentence
2. No market impact; the capital invested by the agent is not significant enough to move the market.
Data
- Front‑month TTF Natural Gas Futures market-prices unknown
Source sentence
The dataset comprises trades of the front‑month contracts of the TTF Natural Gas Futures, which represent the most liquid market, spanning from 2011 to 2022.
Costs
Modeled
A transaction‑cost fraction λc (identical for buying and selling) is subtracted from gross returns; λc is treated as a constant cost parameter.
Source sentence
where λη ∈[0, 1] is the transaction cost fraction that is assumed to be identical for buying and selling.
Finds 3
- The deep RL models achieve an average out‑of‑sample Sharpe ratio that is 83% higher than the buy‑and‑hold baseline. performance
Source sentence
Averaged across all deep RL models and risk‑sensitivity terms, they yield a Sharpe ratio that is 83% higher than the baseline.
- Actor‑based policy‑gradient algorithms outperform actor‑critic algorithms in 7 out of 8 model‑risk‑sensitivity combinations. performance
Source sentence
Comparing two algorithms employing the same network architecture and risk sensitivity term reveals that the actor‑based algorithm outperforms the actor‑critic‑based algorithm in 7 out of 8 combinations.
- CNN‑based models show a slight performance advantage over LSTM‑based models. performance
Source sentence
The actor‑based models outperform the actor‑critic‑based models, while the CNN‑based models show a slight performance advantage over the LSTM‑based models.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.