Deep Q-Learning

Application of Deep Reinforcement Learning to At-the-Money S&P 500 Options Hedging ArXiv ID: 2510.09247 “View on arXiv” Authors: Zofia Bracha, Paweł Sakowski, Jakub Michańków Abstract This paper explores the application of deep Q-learning to hedging at-the-money options on the S&P500 index. We develop an agent based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, trained to simulate hedging decisions without making explicit model assumptions on price dynamics. The agent was trained on historical intraday prices of S&P500 call options across years 2004–2024, using a single time series of six predictor variables: option price, underlying asset price, moneyness, time to maturity, realized volatility, and current hedge position. A walk-forward procedure was applied for training, which led to nearly 17~years of out-of-sample evaluation. The performance of the deep reinforcement learning (DRL) agent is benchmarked against the Black–Scholes delta-hedging strategy over the same period. We assess both approaches using metrics such as annualized return, volatility, information ratio, and Sharpe ratio. To test the models’ adaptability, we performed simulations across varying market conditions and added constraints such as transaction costs and risk-awareness penalties. Our results show that the DRL agent can outperform traditional hedging methods, particularly in volatile or high-cost environments, highlighting its robustness and flexibility in practical trading contexts. While the agent consistently outperforms delta-hedging, its performance deteriorates when the risk-awareness parameter is higher. We also observed that the longer the time interval used for volatility estimation, the more stable the results. ...

Efficient Triangular Arbitrage Detection via Graph Neural Networks ArXiv ID: 2502.03194 “View on arXiv” Authors: Unknown Abstract Triangular arbitrage is a profitable trading strategy in financial markets that exploits discrepancies in currency exchange rates. Traditional methods for detecting triangular arbitrage opportunities, such as exhaustive search algorithms and linear programming solvers, often suffer from high computational complexity and may miss potential opportunities in dynamic markets. In this paper, we propose a novel approach to triangular arbitrage detection using Graph Neural Networks (GNNs). By representing the currency exchange network as a graph, we leverage the powerful representation and learning capabilities of GNNs to identify profitable arbitrage opportunities more efficiently. Specifically, we formulate the triangular arbitrage problem as a graph-based optimization task and design a GNN architecture that captures the complex relationships between currencies and exchange rates. We introduce a relaxed loss function to enable more flexible learning and integrate Deep Q-Learning principles to optimize the expected returns. Our experiments on a synthetic dataset demonstrate that the proposed GNN-based method achieves a higher average yield with significantly reduced computational time compared to traditional methods. This work highlights the potential of using GNNs for solving optimization problems in finance and provides a promising approach for real-time arbitrage detection in dynamic financial markets. ...

A Comparative Analysis of Portfolio Optimization Using Mean-Variance, Hierarchical Risk Parity, and Reinforcement Learning Approaches on the Indian Stock Market ArXiv ID: 2305.17523 “View on arXiv” Authors: Unknown Abstract This paper presents a comparative analysis of the performances of three portfolio optimization approaches. Three approaches of portfolio optimization that are considered in this work are the mean-variance portfolio (MVP), hierarchical risk parity (HRP) portfolio, and reinforcement learning-based portfolio. The portfolios are trained and tested over several stock data and their performances are compared on their annual returns, annual risks, and Sharpe ratios. In the reinforcement learning-based portfolio design approach, the deep Q learning technique has been utilized. Due to the large number of possible states, the construction of the Q-table is done using a deep neural network. The historical prices of the 50 premier stocks from the Indian stock market, known as the NIFTY50 stocks, and several stocks from 10 important sectors of the Indian stock market are used to create the environment for training the agent. ...