Paper: arXiv 2504.11258

Authors: Liam Welsh, Udit Grover, Sebastian Jaimungal

Abstract

Climate change is a major threat to the future of humanity, and its impacts are being intensified by excess man-made greenhouse gas emissions. One method governments can employ to control these emissions is to provide firms with emission limits and penalize any excess emissions above the limit. Excess emissions may also be offset by firms who choose to invest in carbon reducing and capturing projects. These projects generate offset credits which can be submitted to a regulating agency to offset a firm’s excess emissions, or they can be traded with other firms. In this work, we characterize the finite-agent Nash equilibrium for offset credit markets. As computing Nash equilibria is an NP-hard problem, we utilize the modern reinforcement learning technique Nash-DQN to efficiently estimate the market’s Nash equilibria. We demonstrate not only the validity of employing reinforcement learning methods applied to climate themed financial markets, but also the significant financial savings emitting firms may achieve when abiding by the Nash equilibria through numerical experiments.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper applies multi-agent reinforcement learning (Nash-DQN) to solve NP-hard finite-agent Nash equilibria in greenhouse gas offset credit markets. It features a solid theoretical foundation in stochastic games combined with numerical experiments based on Canadian carbon pricing regulations.

Research Flowchart

  flowchart TD
    A[Research Goal: Model GHG Offset Credit Markets & Find Nash Equilibrium] --> B[Input Data & Setup: Emission Limits, Penalties, Project Costs & Credit Trading]
    B --> C[Methodology: Multi-Agent Reinforcement Learning MARL]
    C --> D[Computational Process: Nash-DQN Algorithm to Estimate Equilibrium]
    D --> E[Key Outcomes: Proved RL Validity in Carbon Markets & Showed Significant Financial Savings]

Cited References (77)

  1. Deep learning for principal-agent mean field games (2021) — link
  2. Estimating Corporate Greenhouse Gas Emissions (2025)
  3. Large-population cost-coupled LQG problems with nonuniform agents: individual-mass behavior and decentralized $\epsilon$- N ash equilibria (2007)
  4. Optimal dynamic regulation of carbon emissions market (2023)
  5. A groundwater market model — → 2501.14071
  6. V A -learning as a more efficient alternative to Q -learning (2023)
  7. Optimal stochastic control for carbon emission with allowances trading (2025)
  8. The complexity of computing a Nash equilibrium (2009)
and 69 more
  1. Greenhouse-Gas-Emission-Aware Portfolio Optimization with Deep Reinforcement Learning (2024)
  2. Large population stochastic dynamic games: closed-loop M c K ean- V lasov systems and the N ash certainty equivalence principle (2006)
  3. Principal agent mean field games in R E C markets (2021) — link
  4. Continuous-time stochastic control and optimization with financial applications (2009)
  5. Optimal Generation and Trading in Solar Renewable Energy Certificate ( S R E C ) Markets (2020)
  6. The Evolution of Canada’s Carbon Markets and Their Role in Energy Transition (2022)
  7. Recent Developments in Machine Learning Methods for Stochastic Control and Games (2022)
  8. Risk-neutral pricing of financial instruments in emission markets: A structural approach (2012)
  9. Why Countries Must Cooperate on Carbon Prices (2022)
  10. Q-learning (1992)
  11. The evolving role of greenhouse gas emission offsets in combating climate change (2024)
  12. Equilibrium in a stochastic $ n $-person game (1964)
  13. Canadian Greenhouse Gas Offset Credit System Regulations: S O R /2022-111 (2022)
  14. Carbon pollution pricing systems across C anada (2023)
  15. How to purchase carbon emission right optimally for energy-consuming enterprises? A nalysis based on optimal stopping model (2023)
  16. Selling carbon offsets to the P rovince (2024)
  17. Offset Credits (2024)
  18. C anada’s Greenhouse Gas Offset Credit System Public Registry (2024)
  19. A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems (2005)
  20. Continuous deep q-learning with model-based acceleration (2016)
  21. Principal-Agent Reinforcement Learning (2024) — link
  22. Nash Q -learning for general-sum stochastic games (2003)
  23. A course in game theory (1994)
  24. Deep Q -learning for N ash equilibria: Nash- D Q N (2022)
  25. Canada’s Greenhouse Gas Offset Credit System (2022)
  26. The federal carbon pollution pricing benchmark (2022)
  27. A Forward-Backward SDEs Approach to Pricing in Carbon Markets (2017)
  28. Modelling complex systems of heterogeneous agents to better design sustainability transitions policy (2016)
  29. The impact of heterogeneous market players with bounded-rationality on the electricity sector low-carbon transition (2020)
  30. The valuation of clean spread options: linking electricity, emissions and fuels (2012)
  31. Optimal stochastic control and carbon price formation (2009)
  32. A class of games possessing pure-strategy Nash equilibria (1973)
  33. Anonymous sequential games (1988)
  34. Bertrand and Cournot mean field games (2015)
  35. Mean field games and oil production (2010)
  36. Price volatility and banking in green certificate markets (2006)
  37. Tradable green certificates for renewable support: The role of expectations and uncertainty (2017)
  38. Price formation and optimal trading in intraday electricity markets (2022)
  39. An extended mean field game for storage in smart grids (2020)
  40. Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle (2006)
  41. Mean field games (2007)
  42. State and trends of carbon pricing 2021 (2021)
  43. The Carbon Equivalence Principle: Methods and Applications (2022)
  44. The carbon equivalence principle: minimizing the cost to carbon net zero (2023)
  45. An Uncertainty-based Risk Management Framework for Climate-Change Risk (2022)
  46. Probabilistic analysis of mean-field games (2013)
  47. Advanced fixed point theory for economics (2018)
  48. Learning from delayed rewards (1989)
  49. The Financial Impact of Carbon Emissions on Power Utilities Under Climate Scenarios (2024)
  50. ‘ C limate-Controlled’ Sausage? C ourts Crack Down on ‘Greenwashing’ (2024)
  51. Carbon pollution pricing systems across C anada (2021)
  52. A case study on stochastic games on large graphs in mean field and sparse regimes (2021)
  53. Nash equilibria in greenhouse gas offset credit markets — → 2401.01427
  54. A Maximum Principle approach to deterministic Mean Field Games of Control with Absorption (2021) — link
  55. A mean-field game approach to equilibrium pricing in solar renewable energy certificate markets (2022)
  56. The Potential Method For Price-Formation Models (2022) — link
  57. Equilibrium price dynamics of emission permits (2018)
  58. Dynamic behavior of C O $_2$ spot prices (2008)
  59. On mean field games models for exhaustible commodities trade (2020)
  60. Reinforcement learning: An introduction (2018)
  61. Adjoint and compensated compactness methods for Hamilton–Jacobi PDE (2010)
  62. Partial differential equations (2022)
  63. An introduction to game theory (2004)
  64. Game theory (1991)
  65. High-dimensional continuous control using generalized advantage estimation (2015) — link
  66. Mean field models to regulate carbon emissions in electricity production (2022)
  67. Geophysical Control of Geological Carbon Storage using Deep Reinforcement Learning: Sensitivity to Multi-Geophysical Noise and to the Uncertainty of Digital Twins (2025)
  68. Can investors curb greenwashing? (2025)
  69. Climate impact investing (2023)

2 of 77 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.