Paper: arXiv 2309.00630

Authors: Jonas Hanetho

Abstract

Algorithmic trading has gained attention due to its potential for generating superior returns. This paper investigates the effectiveness of deep reinforcement learning (DRL) methods in algorithmic commodities trading. It formulates the commodities trading problem as a continuous, discrete-time stochastic dynamical system. The proposed system employs a novel time-discretization scheme that adapts to market volatility, enhancing the statistical properties of subsampled financial time series. To optimize transaction-cost- and risk-sensitive trading agents, two policy gradient algorithms, namely actor-based and actor-critic-based approaches, are introduced. These agents utilize CNNs and LSTMs as parametric function approximators to map historical price observations to market positions.Backtesting on front-month natural gas futures demonstrates that DRL models increase the Sharpe ratio by $83%$ compared to the buy-and-hold baseline. Additionally, the risk profile of the agents can be customized through a hyperparameter that regulates risk sensitivity in the reward function during the optimization process. The actor-based models outperform the actor-critic-based models, while the CNN-based models show a slight performance advantage over the LSTM-based models.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper uses advanced stochastic calculus and deep RL theory for continuous action spaces and policy gradients, while providing backtested results on natural gas futures with specific performance metrics.

Research Flowchart

  flowchart TD
  A["<b>Research Goal</b><br>Effectiveness of DRL in<br>Commodities Trading?"] --> B["<b>Methodology</b><br>Novel Volatility-Adaptive<br>Time-Discretization"]
  B --> C["<b>Agent Optimization</b><br>Policy Gradient Algorithms<br>Actor vs. Actor-Critic"]
  C --> D["<b>Computational Process</b><br>CNN & LSTM Function<br>Approximators"]
  D --> E["<b>Implementation</b><br>Backtesting on<br>Front-Month Natural Gas Futures"]
  E --> F["<b>Key Findings</b><br>83% ↑ Sharpe Ratio<br>Actor > Actor-Critic<br>Customizable Risk Profile"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a deep reinforcement learning framework for commodities trading that includes a novel time‑discretization scheme adaptive to market volatility and two policy‑gradient algorithms (actor‑based and actor‑critic‑based) using CNN and LSTM networks.

Source sentence
It formulates the commodities trading problem as a continuous, discrete-time stochastic dynamical system. The proposed system employs a novel time-discretization scheme that adapts to market volatility, enhancing the statistical properties of subsampled financial time series.

Assumes 2

  • No slippage; all trades can be executed at the exact quoted price. execution
    Source sentence
    1. No slippage; all trades can be executed at the exact quoted price.
  • No market impact; the capital invested by the agent is not significant enough to move the market. market
    Source sentence
    2. No market impact; the capital invested by the agent is not significant enough to move the market.

Data

  • Front‑month TTF Natural Gas Futures market-prices unknown
    2011‑2022 · front‑month contracts of the TTF Natural Gas Futures
    Source sentence
    The dataset comprises trades of the front‑month contracts of the TTF Natural Gas Futures, which represent the most liquid market, spanning from 2011 to 2022.

Costs

Modeled Costs are part of the reported headline results.

A transaction‑cost fraction λc (identical for buying and selling) is subtracted from gross returns; λc is treated as a constant cost parameter.

Source sentence
where λη ∈[0, 1] is the transaction cost fraction that is assumed to be identical for buying and selling.

Finds 3

  • The deep RL models achieve an average out‑of‑sample Sharpe ratio that is 83% higher than the buy‑and‑hold baseline. performance
    Source sentence
    Averaged across all deep RL models and risk‑sensitivity terms, they yield a Sharpe ratio that is 83% higher than the baseline.
  • Actor‑based policy‑gradient algorithms outperform actor‑critic algorithms in 7 out of 8 model‑risk‑sensitivity combinations. performance
    Source sentence
    Comparing two algorithms employing the same network architecture and risk sensitivity term reveals that the actor‑based algorithm outperforms the actor‑critic‑based algorithm in 7 out of 8 combinations.
  • CNN‑based models show a slight performance advantage over LSTM‑based models. performance
    Source sentence
    The actor‑based models outperform the actor‑critic‑based models, while the CNN‑based models show a slight performance advantage over the LSTM‑based models.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.