{"paper_id":"2308.01910","title":"Deep Policy Gradient Methods in Commodity Markets","authors":["Jonas Hanetho"],"paper_date":"2023-06-14","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2308.01910v1","page_url":"https://thequant.space/flowcharts/deep-policy-gradient-methods-in-commodity-markets/","scores":{"empirical_rigor":7,"math_complexity":8,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes two policy‑gradient algorithms (an actor‑based and an actor‑critic‑based) and a novel reactive, adaptive time‑discretization scheme for commodity trading.","quote":"Two policy gradient algorithms, an actor-based and an actor-critic-based, are proposed to optimize a transaction-cost- and risk-sensitive agent.","summary":"The energy transition has increased the reliance on intermittent energy sources, destabilizing energy markets and causing unprecedented volatility, culminating in the global energy crisis of 2021. In addition to harming producers and consumers, volatile energy markets may jeopardize vital…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"No slippage: sufficient liquidity to fill any order at the quoted price.","type":"execution","stated":"explicit","quote":"No slippage, i.e., there is sufficient liquidity in the market to fill any orders placed by the agent, regardless of size, at the quoted price."},{"assumption":"No market impact: the agent’s trades are too small to move the market.","type":"market","stated":"explicit","quote":"No market impact, i.e., the money invested by the agent is not significant enough to move the market."},{"assumption":"Fractional trading assumption: positions can be scaled arbitrarily, which is less reasonable with small capital.","type":"other","stated":"explicit","quote":"The fractional trading assumption is less reasonable the smaller the amount of capital available to the agent."}]},"datasets":{"status":"available","items":[{"name":"front‑month TTF Natural Gas Futures contracts","provider":null,"kind":"market-prices","access":"unknown","period":"2011-2022","universe":"Natural Gas futures","quote":"The dataset consists of the front‑month TTF Natural Gas Futures contracts from 2011 to 2022."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"intraday-bars"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":null,"detail":"A transaction‑cost fraction λc is included in the reward function and set to a reasonable value reflecting market conditions.","quote":"∈[0,1] is the transaction cost fraction that is assumed to be identical for buying and selling.","source":"front_matter","rule":"R3","candidates":{"front_matter":"yes","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"A transaction‑cost fraction λc is included in the reward function and set to a reasonable value reflecting market conditions.","card_quote":"∈[0,1] is the transaction cost fraction that is assumed to be identical for buying and selling.","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The deep RL agents achieve an 83 % higher Sharpe ratio than the buy‑and‑hold baseline on out‑of‑sample backtests.","kind":"performance","quote":"On average, the deep reinforcement learning agents produce an 83% higher Sharpe ratio out‑of‑sample than the buy‑and‑hold baseline."},{"finding":"The actor‑based policy gradient algorithm yields about 49 % higher Sharpe on average compared to the deterministic actor‑critic algorithm.","kind":"performance","quote":"algorithm produces significantly higher Sharpe (49% on average) than the deterministic actor‑critic algorithm, suggesting that an actor‑based policy gradient method is more suited to algorithmic trading in an online, continuous time setting."},{"finding":"CNN‑based models obtain roughly 5 % higher Sharpe than LSTM‑based models, possibly due to vanishing‑gradient issues in LSTMs.","kind":"performance","quote":"The parametric function approximator based on the CNN architecture performs slightly better (5% higher Sharpe on average) than the LSTM, possibly due to the problem of vanishing gradients for the LSTM."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":20,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:25:45Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}