Paper: arXiv 2307.09631

Authors: Eduardo C. Garrido-Merchán, Sol Mora-Figueroa-Cruz-Guzmán, María Coronado-Vaca

Abstract

This paper investigates the application of Deep Reinforcement Learning (DRL) for Environment, Social, and Governance (ESG) financial portfolio management, with a specific focus on the potential benefits of ESG score-based market regulation. We leveraged an Advantage Actor-Critic (A2C) agent and conducted our experiments using environments encoded within the OpenAI Gym, adapted from the FinRL platform. The study includes a comparative analysis of DRL agent performance under standard Dow Jones Industrial Average (DJIA) market conditions and a scenario where returns are regulated in line with company ESG scores. In the ESG-regulated market, grants were proportionally allotted to portfolios based on their returns and ESG scores, while taxes were assigned to portfolios below the mean ESG score of the index. The results intriguingly reveal that the DRL agent within the ESG-regulated market outperforms the standard DJIA market setup. Furthermore, we considered the inclusion of ESG variables in the agent state space, and compared this with scenarios where such data were excluded. This comparison adds to the understanding of the role of ESG factors in portfolio management decision-making. We also analyze the behaviour of the DRL agent in IBEX 35 and NASDAQ-100 indexes. Both the A2C and Proximal Policy Optimization (PPO) algorithms were applied to these additional markets, providing a broader perspective on the generalization of our findings. This work contributes to the evolving field of ESG investing, suggesting that market regulation based on ESG scoring can potentially improve DRL-based portfolio management, with significant implications for sustainable investing strategies.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)

Why this score: The paper employs complex mathematical concepts from reinforcement learning (MDPs, A2C, PPO) and deep learning, with detailed formulations in the provided excerpt, while also demonstrating high empirical rigor through multiple backtests across DJIA, IBEX 35, and NASDAQ-100 using standardized platforms like FinRL.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Investigate DRL for ESG Portfolio Management<br>and ESG Market Regulation Impact"] --> B["Methodology:<br>Advantage Actor-Critic (A2C) & PPO<br>via FinRL / OpenAI Gym"]
  B --> C["Data/Inputs:<br>1. Market Indices (DJIA, IBEX 35, NASDAQ-100)<br>2. ESG Scores (Regulated vs. Standard Markets)"]
  C --> D["Computational Process:<br>Training Agents in<br>Standard & ESG-Regulated Environments"]
  D --> E["Key Findings:<br>1. DRL agents outperform in ESG-regulated markets<br>2. ESG integration improves decision-making<br>3. Strategy generalizes across indices"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a deep reinforcement learning framework that incorporates ESG information and market regulation based on ESG scores for financial portfolio management.

Source sentence
We leveraged an Advantage Actor-Critic (A2C) agent and conducted our experiments using environments encoded within the OpenAI Gym, adapted from the FinRL platform.

Assumes 6

  • Training data runs from 2009‑01‑01 to 2020‑07‑01 and trading data from 2020‑07‑01 to 2021‑10‑31. data
    Source sentence
    The training data period for all the DRL agents goes from 2009-01-01 to 2020-07-01 and the trading period goes from 2020-07-01 to 2021-10-31.
  • Missing monthly Bloomberg ESG values are approximated from nearby dates, introducing bias. data
    Source sentence
    As the data is incomplete, we approximate missing ESG values from the existing ones that are closer to the date, incurring in bias that must be targeted in future work.
  • The reward function is modified to include an ESG performance term based on weighted ESG values of the portfolio. model
    Source sentence
    We have considered to modify the reward signal r of the DRL agent to take into account the ESG performance of the portfolio and not only the financial returns.
  • Grants are proportionally allotted to portfolios based on returns and ESG scores, taxes are assigned to portfolios below the mean ESG score of the index. market
    Source sentence
    In the ESG-regulated market, grants were proportionally allotted to portfolios based on their returns and ESG scores, while taxes were assigned to portfolios below the mean ESG score of the index.
Show 2 more
  • The ESG value of the index (ψ) is computed assuming equal weights (1/A) for all assets. model
    Source sentence
    Having computed φ, we now compute φ assuming that every value of the weights wti is 1/A, where A is the total number of assets and denote this quantity as ψ, that represents the ESG value of the index.
  • Hyper‑parameters (e.g., learning rate η = 2e‑4, entropy coefficient c = 5e‑3) are fixed and not tuned in the reported experiments. execution
    Source sentence
    We use 50000 timesteps, a learning rate of η = 2e−4 and an entropy coefficient of c = 5e−3.

Data

  • Monthly Bloomberg ESG information · Bloomberg alternative commercial
    Source sentence
    we have gained access to monthly Bloomberg ESG information about different assets.
  • Daily financial data from Yahoo Finance · Yahoo Finance API market-prices public
    2009-01-01 to 2021-10-31
    Source sentence
    we use the Yahoo Finance API to download the dialy financial data from 2009-01-01 to 2021-10-31.
  • Dow Jones Industrial Average (DJIA) index benchmark public
    DJIA constituents
    Source sentence
    ... comparative analysis of DRL agent performance under standard Dow Jones Industrial Average (DJIA) market conditions
  • IBEX‑35 index benchmark public
    IBEX‑35 constituents
    Source sentence
    We also analyze the behaviour of the DRL agent in IBEX 35 and NASDAQ-100 indexes.
  • NASDAQ‑100 index benchmark public
    NASDAQ‑100 constituents
    Source sentence
    We also analyze the behaviour of the DRL agent in IBEX 35 and NASDAQ-100 indexes.

Costs

Not modeled A trading or execution result is reported without costs.

Finds 4

  • In the ESG‑regulated market the A2C agent achieves higher annual return (0.462587) and cumulative return (0.173427) than in the unregulated DJIA market (0.435209 and 0.164137). performance
    Source sentence
    Metric name DRL A2C Agent ESG Regulated DRL A2C Agent DJIA
  • The ESG‑regulated A2C agent attains a higher Sharpe ratio (2.441398) compared to the unregulated baseline (2.340178). performance
    Source sentence
    Sharpe ratio 2.441398 2.340178 2.127074
  • Including ESG variables in the state space does not degrade performance; agents with ESG information perform comparably to those without. performance
    Source sentence
    Remarkably, we found that including ESG information into the state space of the agents did not adversely affect the performance of the portfolio.
  • Statistical tests across algorithms and indexes fail to reject the null hypothesis that ESG‑aware and ESG‑agnostic policies deliver the same results. statistical
    Source sentence
    we can not reject the null hypothesis that both policies deliver the same results, in statistical terms.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 21 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (2 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.