{"paper_id":"2504.05521","title":"Deep Reinforcement Learning Algorithms for Option Hedging","authors":["Andrei Neagu","Frédéric Godin","Leila Kosseim"],"paper_date":"2025-04-07","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2504.05521v2","page_url":"https://thequant.space/flowcharts/deep-reinforcement-learning-algorithms-for-option-hedging/","scores":{"empirical_rigor":6,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper provides a benchmark comparison of eight deep reinforcement learning algorithms for dynamic option hedging, including two novel variants.","quote":"In this paper, we compare the performance of eight DRL algorithms in the context of dynamic hedging; Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), along with four variants of Deep Q-Learning (DQL) and two variants of Deep Deterministic Policy Gradient (DDPG). Two of these variants represent a ","summary":"Dynamic hedging is a financial strategy that consists in periodically transacting one or multiple financial assets to offset the risk associated with a correlated liability. Deep Reinforcement Learning (DRL) algorithms have been used to find optimal solutions to dynamic hedging problems by framing…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Log‑returns follow a GJR‑GARCH(1,1) process with parameters estimated from monthly S&P 500 data.","type":"model","stated":"explicit","quote":"are learned through maximum likelihood estimates from monthly S&P 500 prices from 2000/11/15 to 2024/10/15."},{"assumption":"Risk‑free rate is set to zero.","type":"market","stated":"explicit","quote":"The risk-free rate is set to rf = 0."},{"assumption":"The hedging environment provides a single reward only at the final time step (sparse rewards).","type":"execution","stated":"explicit","quote":"the formulation of the option hedging problem leads to an environment in which rewards are sparse, where only a single reward is obtained from the environment at the final time step."},{"assumption":"Baseline Black‑Scholes delta hedge assumes no transaction costs and continuous‑time trading under geometric Brownian motion.","type":"model","stated":"explicit","quote":"In the absence of transaction costs, in continuous time, and for market dynamics following a geometric Brownian motion, this procedure is shown to completely eliminate risk."},{"assumption":"Training is performed on simulated price paths rather than real market execution.","type":"data","stated":"explicit","quote":"To evaluate the performance of the different DRL algorithms, we trained them on simulated price paths following Eq. 7..."},{"assumption":"Hyperparameter search is limited due to computational constraints.","type":"other","stated":"explicit","quote":"The main limitation common to most DRL algorithms and also present in our work is their heavy computational load coupled with their dependence on hyperparameters, forcing the hyperparameter search to be narrow."}]},"datasets":{"status":"available","items":[{"name":"Simulated price paths (GJR‑GARCH)","provider":null,"kind":"synthetic","access":"public","period":"2000/11/15 to 2024/10/15 (parameter esti","universe":"S&P 500 (monthly prices for parameter estimation)","quote":"To evaluate the performance of the different DRL algorithms, we trained them on simulated price paths following Eq. 7, where the parameters of the GJR‑GARCH(1,1) model"}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":null,"detail":null,"quote":"In the absence of transaction costs, in continuous time, and for market dynamics following a geometric Brownian motion, this procedure is shown to completely eliminate risk.","source":"front_matter","rule":"R3","candidates":{"front_matter":"no","evidence_card":"no","evidence_card_quoted":true},"card_detail":null,"card_quote":"In the absence of transaction costs, in continuous time, and for market dynamics following a geometric Brownian motion, this procedure is shown to completely eliminate risk.","meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"pricing-calibration","findings":[{"finding":"Monte Carlo Policy Gradient (MCPG) achieves the lowest RSQP (0.8111) and is the only algorithm to outperform the Black‑Scholes delta hedge baseline (0.9038) within the computational budget.","kind":"performance","quote":"MCPG is the only algorithm that was capable of significantly outperforming the B‑S DH baseline in the allotted computational budget (0.8111 vs 0.9038)."},{"finding":"Proximal Policy Optimization (PPO) follows MCPG in performance but does not beat the baseline.","kind":"performance","quote":"Results show that MCPG, followed by PPO, obtain the best performance in terms of the root semi‑quadratic penalty."},{"finding":"MCPG’s training time is more than fourteen times smaller than that of the next best algorithm.","kind":"performance","quote":"Additionally, the training time of MCPG is more than fourteen times smaller than the closest competitor."},{"finding":"Value‑based algorithms (DQL, DDPG variants) perform poorly due to sparse rewards and do not improve over the baseline.","kind":"negative","quote":"the formulation of the option hedging problem leads to an environment in which rewards are sparse."},{"finding":"TD3 marginally lowers RSQP compared to vanilla DDPG but still fails to beat the baseline and requires significantly more training time.","kind":"performance","quote":"This seems to only marginally lower the RSQP over vanilla DDPG (1.0113 vs 1.0467)"}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[],"contradicts":[],"qualifies":[],"compared_with":51,"citations":{"cited_by_in_corpus":2,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":1,"mentions":0,"unclassified":1},"independent_papers":[{"paper":{"paper_id":"2510.09247","title":"Application of Deep Reinforcement Learning to At-the-Money S&P 500 Options Hedging","paper_date":"2025-10-10","authors":["Zofia Bracha","Paweł Sakowski","Jakub Michańków"],"empirical_rigor":7.5,"page_url":"https://thequant.space/flowcharts/application-of-deep-reinforcement-learning-to-at-the-money-s/"},"how":"discusses_findings","quote":"Along related lines, [CITED] apply deep hedging under a GJR–GARCH(1,1) model and benchmark a range of reinforcement learning algorithms. They find that Monte Carlo policy-gradient methods consistently outperform value-based approaches in terms of hedging error, with policy-gradient methods being the only ones to surpass the Black–Scholes delta baseline."}],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:41:38Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}