{"paper_id":"2510.09247","title":"Application of Deep Reinforcement Learning to At-the-Money S&P 500 Options Hedging","authors":["Zofia Bracha","Paweł Sakowski","Jakub Michańków"],"paper_date":"2025-10-10","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2510.09247v1","page_url":"https://thequant.space/flowcharts/application-of-deep-reinforcement-learning-to-at-the-money-s/","scores":{"empirical_rigor":7.5,"math_complexity":8.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a deep reinforcement learning agent based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm for hedging at‑the‑money S&P 500 options.","quote":"We develop an agent based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, trained to simulate hedging decisions without making explicit model assumptions on price dynamics.","summary":"This paper explores the application of deep Q-learning to hedging at-the-money options on the S&P500 index. We develop an agent based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, trained to simulate hedging decisions without making explicit model assumptions on price…","strategy":"TD3 deep reinforcement learning hedging agent"},"assumptions":{"status":"available","items":[{"assumption":"A constant risk‑free rate of 2% is used for all periods.","type":"market","stated":"explicit","quote":"Across all periods we assumed 2% risk free rate."},{"assumption":"Rebalancing occurs every 30 minutes.","type":"execution","stated":"explicit","quote":"current setup of trading every 30 minutes drastically increase the costs."},{"assumption":"Realized volatility is estimated using a rolling standard deviation of log‑returns.","type":"data","stated":"explicit","quote":"In this paper we employ a realized volatility framework . First, asset prices"},{"assumption":"A walk‑forward training procedure is used to generate nearly 17 years of out‑of‑sample evaluation.","type":"other","stated":"explicit","quote":"A walk‑forward procedure was applied for training, which lead to nearly 17 years of out‑of‑sample evaluation."},{"assumption":"The benchmark delta‑hedging strategy assumes continuous trading with no transaction costs or liquidity constraints (Black‑Scholes model).","type":"model","stated":"explicit","quote":"It assumes a market with constant volatility and interest rates, allowance of continuous trading without transaction costs or liquidity constraints."}]},"datasets":{"status":"available","items":[{"name":"Historical intraday prices of S&P 500 call options","provider":null,"kind":"options","access":"unknown","period":"2004‑2024","universe":"at‑the‑money S&P 500 call options","quote":"The agent was trained on historical intraday prices of S&P 500 call options across years 2004 to 2024, using a single time series of six predictor variables: option price, underlying asset price, moneyness, time to maturity, realized volatility, and current hedge position."}],"known":[{"id":"cboe","name":"Cboe","provider":"Cboe Global Markets","kind":"options","access":"commercial","quote":"The data was sourced from CBOE DataShop.","mentions":1,"used_in":1}],"sample":{"start":"2004","end":"2024","market":"S&P 500 options","asset_class":"options","frequency":"intraday"},"data_resolution":"intraday-bars"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":true,"detail":"Costs are included via a flat parameter c (e.g., c = 0.1% in the base case) in the PnL calculation.","quote":"c represents transaction costs, HS holding of stock, S stock price, HO holding of option, and finally C represents option price.","source":"front_matter","rule":"R3","candidates":{"front_matter":"yes","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"Costs are included via a flat parameter c (e.g., c = 0.1% in the base case) in the PnL calculation.","card_quote":"c represents transaction costs, HS holding of stock, S stock price, HO holding of option, and finally C represents option price.","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The DRL agent outperforms the Black‑Scholes delta‑hedging benchmark across all reported metrics in the base case.","kind":"performance","quote":"The results in the Table 2 show superior performance of the DRL agent over the classical delta hedging benchmark model. The DRL managed to get positive Information ratio, despite heavy drawdowns and negative returns."},{"finding":"When transaction costs are reduced, the DRL agent’s performance improves and can become positive, while the benchmark may still lag.","kind":"performance","quote":"When the costs are low, the benchmark performs better both in terms of returns or information ratio, but also experiences lower drawdowns."},{"finding":"Higher risk‑awareness (risk‑penalty parameter) reduces the DRL agent’s returns and performance.","kind":"performance","quote":"During the sensitivity analysis, we observed that the increase in risk aversion reduces returns, while shorter volatility estimation intervals make the model more responsive to jumps"},{"finding":"The DRL framework shows particular advantage in volatile or high‑cost environments, adapting to market conditions where delta hedging deteriorates sharply.","kind":"performance","quote":"Our results show that the DRL agent can outperform traditional hedging methods, particularly in volatile or high‑cost environments, highlighting its robustness and flexibility in practical trading contexts."}],"reported_performance":{"strategy":"TD3 deep reinforcement learning hedging agent","sharpe":-0.417,"sharpe_benchmark":-0.418,"annual_return_pct":-0.4,"total_return_pct":-45.51,"max_drawdown_pct":75.26},"out_of_sample":{"method":"walk-forward","headline_out_of_sample":true}},"independent_evidence":{"status":"available","supports":[{"paper":{"paper_id":"2508.16595","title":"Empirical Analysis of the Model-Free Valuation Approach: Hedging Gaps, Conservatism, and Trading Opportunities","paper_date":"2025-08-09","authors":["Zixing Chen","Yihan Qi","Shanlan Que","Julian Sester","Xiao Zhang"],"empirical_rigor":8,"page_url":"https://thequant.space/flowcharts/empirical-analysis-of-the-model-free-valuation-approach-hed/"},"question":"How does increasing the risk‑parameter (c or risk‑penalty) affect average profit and risk‑adjusted returns?","this_paper_finding":"Higher risk‑awareness (risk‑penalty parameter) reduces the DRL agent’s returns and performance.","other_paper_finding":"Increasing the risk parameter c reduces average profit and risk‑adjusted returns.","other_paper_quote":"with an increasing c the average profit decreases due to the increasing costs of the strategy.","note":"Both findings report that higher risk‑parameter reduces returns: A shows c lowers average profit and risk‑adjusted returns; B shows risk‑penalty lowers DRL agent returns. Markets differ (model‑free valuation vs DRL on AT","confidence":0.723},{"paper":{"paper_id":"2305.18941","title":"A Game of Competition for Risk","paper_date":"2023-05-30","authors":["Louis Abraham"],"empirical_rigor":6.5,"page_url":"https://thequant.space/flowcharts/a-game-of-competition-for-risk/"},"question":"How does increasing a risk‑penalty affect average risk taken and resulting returns?","this_paper_finding":"Higher risk‑awareness (risk‑penalty parameter) reduces the DRL agent’s returns and performance.","other_paper_finding":"Penalties reduce both the average risk taken by players and their total rewards.","other_paper_quote":"We find that penalties reduce both the average risk taken by players and their total rewards.","note":"Both findings show that higher risk‑penalties lower the average risk and also reduce total rewards or performance.","confidence":0.499}],"contradicts":[],"qualifies":[],"compared_with":48,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/zof-br/DRL_hedging","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:24:58Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}