Paper: arXiv 2511.04515

Authors: Mathieu Laurière, Ariel Neufeld, Kyunghyun Park

Abstract

We propose and analyze a framework for discrete-time robust mean-field control problems under common noise uncertainty. In this framework, the mean-field interaction describes the collective behavior of infinitely many cooperative agents’ state and action, while the common noise – a random disturbance affecting all agents’ state dynamics – is uncertain. A social planner optimizes over open-loop controls on an infinite horizon to maximize the representative agent’s worst-case expected reward, where worst-case corresponds to the most adverse probability measure among all candidates inducing the unknown true law of the common noise process. We refer to this optimization as a robust mean-field control problem under common noise uncertainty. We first show that this problem arises as the asymptotic limit of a cooperative $N$-agent robust optimization problem, commonly known as propagation of chaos. We then prove the existence of an optimal open-loop control by linking the robust mean field control problem to a lifted robust Markov decision problem on the space of probability measures and by establishing the dynamic programming principle and Bellman–Isaac fixed point theorem for the lifted robust Markov decision problem. Finally, we complement our theoretical results with numerical experiments motivated by distribution planning and systemic risk in finance, highlighting the advantages of accounting for common noise uncertainty.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper provides a rigorous mathematical theoretical foundation for discrete-time robust mean-field control with common noise uncertainty, complete with propagation of chaos and Bellman-Isaacs fixed-point proofs. However, as a predominantly theoretical applied math/quantitative finance paper, its empirical rigor relies on illustrative numerical toy models rather than empirical financial backtesting.

Research Flowchart

  flowchart TD
    A["Research Goal:<br>Formulate robust mean-field control<br>under common noise uncertainty"] --> B["N-Agent Optimization Problem:<br>Cooperative agents subject to<br>shared uncertain disturbances"]
    
    B --> C["Propagation of Chaos:<br>Prove convergence from N-agent<br>to mean-field limit (N → ∞)"]
    
    C --> D["Lifted Robust MDP:<br>Map to probability measure space &<br>apply Dynamic Programming Principle"]
    
    D --> E["Bellman-Isaacs Fixed Point Theorem:<br>Establish existence of optimal<br>open-loop control policy"]
    
    F["Data / Inputs:<br>Financial Systemic Risk &<br>Distribution Planning Scenarios"] --> G["Numerical Experiments:<br>Evaluate robust control vs.<br>non-robust baselines"]
    
    E --> G
    
    G --> H["Key Outcome:<br>Mitigates systemic risk and improves<br>worst-case expected rewards"]

Cited References (130)

  1. Scalable Task-Driven Robotic Swarm Control via Collision Avoidance and Learning Mean-Field Control (2023)
  2. Robustness (2008)
  3. Policy optimization for linear-quadratic zero-sum mean-field type games (2020)
  4. Linear-quadratic zero-sum mean-field type games: O ptimality conditions and policy optimization (2021)
  5. Probability Theory: A Comprehensive Course (2014)
  6. The mathematics of arbitrage (2006)
  7. Limit theorems for stochastic processes (2013)
  8. Multidimensional diffusion processes (1997)
and 122 more
  1. Calcul stochastique et problemes de martingales (1979)
  2. Semimartingale theory and stochastic calculus (2019)
  3. Stochastic Integration and Differential Equations (2005)
  4. Mathematical methods for financial markets (2009)
  5. Infinite dimensional analysis: A Hitchhiker’s Guide (2006)
  6. Convergence of probability measures (2013)
  7. Measure Theory: Volume II (2007)
  8. Markov decision processes under model uncertainty (2023)
  9. Markov– Nash equilibria in mean-field games with discounted cost (2018)
  10. Convergence of Lebesgue integrals with varying measures (1982)
  11. Learning in Mean Field Games: A Survey (2205.12944)
  12. On the convergence of model free learning in mean field games (2020)
  13. Large population stochastic dynamic games: Closed loop McKean–Vlasov sysyems and the Nash certainity equivalence principle (2006)
  14. Mean field games (2007)
  15. Discrete time mean-field stochastic linear-quadratic optimal control problems (2013)
  16. Discrete-time decentralized control using the risk-sensitive performance criterion in the large population regime: a mean field approach (Jul. 2015)
  17. Linear-quadratic-Gaussian mean field games under high rate quantization (Dec. 2013)
  18. Approximate Nash equilibria in partially observed stochastic games with mean-field interactions (2019)
  19. Robust mean field games for coupled Markov jump linear systems (2016)
  20. Linear quadratic risk-sensitive and robust mean field games (2016)
  21. Risk-sensitive mean-field games (2013)
  22. Mean field games for large-population multiagent systems with Markov jump parameters (2012)
  23. Robust mean field games (2016)
  24. Mean field LQG games with model uncertainty (Dec. 2013)
  25. Non-concave distributionally robust stochastic control in a discrete time finite horizon setting — → 2404.05230
  26. Quantifying distributional model risk via optimal transport (2019)
  27. Distributionally robust optimization under moment uncertainty with application to data-driven problems (2010)
  28. Maxmin expected utility with non-unique prior (1989)
  29. Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations (2018)
  30. Mean field games and mean field type control theory (2013)
  31. Probabilistic theory of mean field games with applications I-II (2018)
  32. Discrete time, finite state space mean field games (2010)
  33. Wasserstein distributionally robust stochastic control: A data-driven approach (2020)
  34. Applications of mean field games in financial engineering and economic theory (2020) — link
  35. Mean Field Games and systemic risk (2015)
  36. Mean field games of timing and models for bank runs (2017)
  37. On a mean field game approach modeling congestion and aversion in pedestrian crowds (2011)
  38. Dynamic driving and routing games for autonomous vehicles on networks: A mean field game approach (2021)
  39. Optimal incentives to mitigate epidemics: a Stackelberg mean field game approach (2022)
  40. Contact rate epidemic control of COVID-19 : an equilibrium view (2020)
  41. Ambiguity, risk, and asset returns in continuous time (2002)
  42. Intertemporal Asset Pricing under Knightian Uncertainty (1994)
  43. Uncertainty aversion, risk aversion, and the optimal choice of portfolio (1992)
  44. Notes on mean field games (from P.-L. Lions’ lectures at Colle`ge de France ) (2011)
  45. Mean field games models—a brief survey (2014)
  46. Large-population LQG games involving a major player: the Nash certainty equivalence principle (2010)
  47. A general characterization of the mean field limit for stochastic differential games (2016)
  48. A case study on stochastic games on large graphs in mean field and sparse regimes (2022)
  49. Mean field and n-agent games for optimal investment under relative performance criteria (2019)
  50. FROM THE MASTER EQUATION TO MEAN FIELD GAME LIMIT THEORY (2020)
  51. Convergence of large population games to mean field games with interaction through the controls (2022)
  52. Continuous time, finite state mean field games (2013)
  53. Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter (1970)
  54. Anonymous sequential games (1988)
  55. Robust utility maximization in discrete-time markets with friction (2018)
  56. Utility maximization under model uncertainty in discrete time (2016)
  57. Arbitrage and Duality in Nondominated Discrete-Time Models (2015)
  58. Model-free mean-field reinforcement learning: mean-field MDP and mean-field $Q$- learning (2023)
  59. A mean field approach for optimization in discrete time (2011)
  60. Mean field for Markov decision processes: from discrete to continuous optimization (2012)
  61. Dynamic programming principles for mean-field controls with learning (2023)
  62. Mean-field controls with Q -learning for cooperative MARL : convergence and complexity analysis (2021)
  63. Mean-field Markov decision processes with common noise and open-loop controls (2022)
  64. Mean field Markov decision processes (2023)
  65. Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games I : The ergodic case (2021)
  66. Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: II —the finite horizon case (2022)
  67. Optimal Transport: Old and New (2008)
  68. An extension of Skorohod’s almost;sure representation theorem (1983)
  69. Random measures, theory and applications (2017)
  70. Foundations of modern probability (2002)
  71. Markov-Nash equilibria in mean-field games under model uncertainty (2024) — link
  72. Markov decision processes with applications to finance (2011)
  73. General topology (1989)
  74. On the rate of convergence in Wasserstein distance of the empirical measure (2015)
  75. On the mean speed of convergence of empirical and occupation measures in Wasserstein distance (2014)
  76. A review of probabilistic macroscopic models for swarm robotic systems (2004)
  77. Mean-field models in swarm robotics: A survey (2019)
  78. Continuous time mean-variance portfolio optimization through the mean field approach (2016)
  79. Multi-robot task allocation: A review of the state-of-the-art (2015)
  80. A mean-field control problem of optimal portfolio liquidation with semimartingale strategies (2024)
  81. Deep learning for mean field games and mean field control with applications to finance (2021) — link
  82. A machine learning framework for solving high-dimensional mean field game and mean field control problems (2020)
  83. Mean-field multiagent reinforcement learning: A decentralized network approach (2025)
  84. Extended mean field control problem: A propagation of chaos result (2022)
  85. Limit theory for controlled McKean–Vlasov dynamics (2017)
  86. McKean–Vlasov optimal control: limit theory and equivalence between different formulations (2022)
  87. Dynamic programming for optimal control of stochastic McKean–Vlasov dynamics (2017)
  88. A class of finite-dimensional numerically solvable McKean–Vlasov control problems (2019)
  89. Linear quadratic optimal control of conditional McKean–Vlasov equation with random coefficients and applications (2016)
  90. Mean–field moral hazard for optimal energy demand response management (2021)
  91. Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource (2016)
  92. A weak martingale approach to linear-quadratic McKean–Vlasov stochastic control problems (2019)
  93. Portfolio selection with parameter and model uncertainty: A multi-prior approach (2007)
  94. Quantitative propagation of chaos for mean field Markov decision process with common noise (2023)
  95. Robust $Q$ -learning algorithm for Markov decision processes under Wasserstein uncertainty (2024)
  96. Dynamic programming for mean-field type control (2016)
  97. Randomized dynamic programming principle and Feynman–Kac representation for optimal control of McKean–Vlasov dynamics (2018)
  98. Mean-field optimal control as gamma-limit of finite agent controls (2019)
  99. McKean–Vlasov optimal control: The dynamic programming principle (2022)
  100. Viscosity solutions for McKean–Vlasov control on a torus (2024)
  101. Viscosity solutions for controlled McKean–Vlasov jump-diffusions (2020)
  102. Control on Hilbert spaces and application to some mean field type control problems (2024)
  103. Zero-sum stochastic differential games of generalized McKean–Vlasov type (2019)
  104. Social optima in mean field linear-quadratic-Gaussian control with volatility uncertainty (2021)
  105. Robust mean field linear quadratic social control: Open-loop and closed-loop strategies (2022)
  106. Social optima in robust mean field LQG control: From finite to infinite horizon (2020)
  107. Robust mean field linear-quadratic-Gaussian games with unknown $L^2$ -disturbance (2017)
  108. Robust cooperative multi-agent reinforcement learning: A mean-field type game perspective (2024)
  109. Social optima in robust mean field LQG control (2017)
  110. Social optima in mean field linear-quadratic-Gaussian models with Markov jump parameters (2017)
  111. Optimal solutions to infinite-player stochastic teams and mean-field teams (2020)
  112. Discrete time McKean–Vlasov control problem: A dynamic programming approach (2016)
  113. Infinite horizon average cost optimality criteria for mean-field control (2024)
  114. Distributionally robust Markov decision processes and their connection to risk measures (2022)
  115. Robust solutions to Markov decision problems with uncertain transition matrices (2005)
  116. Distributionally Robust $Q$ -Learning (2022)
  117. Robust Markov decision processes (2013)
  118. Distributionally robust Markov decision processes (2012)
  119. A convex optimization approach to distributionally robust Markov decision processes with Wasserstein distance (2017)
  120. Policy gradient algorithms for robust MDP s with non-rectangular uncertainty sets (2023) — link
  121. Distributionally Robust Deep $Q$ -Learning — → 2505.19058
  122. Nonparametric adaptive robust control under model uncertainty (2023)

2 of 130 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.