Eliciting ESG Preferences for Reinforcement Learning-Based Portfolio Optimization

Paper: arXiv 2609.02677

Authors: Giovanni Dispoto, Marcello Restelli, Carmine Ventre

Abstract

Modern portfolio management increasingly demands a balance between traditional risk-adjusted returns and strict Environmental, Social, and Governance (ESG) mandates. Current Reinforcement Learning (RL) approaches typically optimize for a single ESG provider, neglecting the significant divergence in rating methodologies across the industry and the unintuitive nature of manually weighting conflicting objectives. This paper addresses these limitations by formulating ESG-aware portfolio optimization as a Multi-Objective Reinforcement Learning (MORL) problem that simultaneously incorporates ratings from three distinct ESG agencies. To bridge the gap between high-dimensional algorithmic trade-offs and human decision-making, we integrate a Preference Elicitation framework using Gaussian Processes. This system enables practitioners to infer their latent utility functions through intuitive pairwise comparisons of candidate portfolios based on their Sharpe ratios and aggregate ESG scores. We systematically evaluate our framework by employing Large Language Model (LLM) personas to simulate Portfolio Managers operating under varied regional contexts. Empirical results using historical market data reveal that regional backgrounds fundamentally shift the derived preference weights. For instance, European-based personas tend to prioritize ESG alignment over financial returns, while Texas-based personas favor risk-adjusted performance. This work offers a highly adaptable framework that successfully aligns multi-objective algorithmic trading with diverse, real-world human sustainability preferences.

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents a novel integration of MORL with preference elicitation for ESG portfolio optimization, addressing a critical real-world problem. While the empirical setup uses synthetic ESG data, the methodology is sound and the LLM persona evaluation adds an interesting dimension. The mathematical framework for MORL and Gaussian Processes is well-defined.

Research Flowchart

  flowchart TD
    A[Research Goal: ESG-aware Portfolio Optimization with Diverse Preferences] --> B{Key Methodology Steps};
    B --> C[1. Formulate as MORL (Financial & 3 ESG Agencies)];
    B --> D[2. Integrate Preference Elicitation via Gaussian Processes];
    D --> E[3. Infer Latent Utility Functions via Pairwise Comparisons];
    F[Data/Inputs: Historical Market Data & ESG Ratings] --> C;
    G[Computational Processes: RL Algorithms & GP for Elicitation] --> E;
    H[LLM Personas (Regional Contexts) Simulate PMs] --> E;
    E --> I[Key Findings/Outcomes: Regional Backgrounds Fundamentally Shift Preference Weights];
    I --> J[e.g., European PMs: ESG > Financial; Texas PMs: Financial > ESG];
    I --> K[Outcome: Adaptable Framework Aligns Multi-Objective Trading with Diverse Human Sustainability Preferences];

Cited References (46)

1. Proximal policy optimization algorithms (2017) — link
2. Reinforcement learning: An introduction (2018)
3. Two Tales of Persona in LLM s: A Survey of Role-Playing and Personalization (2024)
4. Aggregate Confusion: The Divergence of ESG Ratings (2022)
5. Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization (2024)
6. Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization (2026)
7. Learning FX trading strategies with FQI and persistent actions (2022)
8. Preferences in artificial intelligence (2016)

1 of 46 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.