Paper: arXiv 2407.04521

Authors: Xiaoli Wei, Xiang Yu, Fengyi Yuan

Abstract

This paper studies the continuous-time q-learning in mean-field jump-diffusion models when the population distribution is not directly observable. We propose the integrated q-function in decoupled form (decoupled Iq-function) from the representative agent’s perspective and establish its martingale characterization, which provides a unified policy evaluation rule for both mean-field game (MFG) and mean-field control (MFC) problems. Moreover, we consider the learning procedure where the representative agent updates the population distribution based on his own state values. Depending on the task to solve the MFG or MFC problem, we can employ the decoupled Iq-function differently to characterize the mean-field equilibrium policy or the mean-field optimal policy respectively. Based on these theoretical findings, we devise a unified q-learning algorithm for both MFG and MFC problems by utilizing test policies and the averaged martingale orthogonality condition. For several financial applications in the jump-diffusion setting, we obtain the exact parameterization of the decoupled Iq-functions and the value functions, and illustrate our q-learning algorithm with satisfactory performance.

Complexity vs Empirical Score

  • Math Complexity: 9.2/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper is highly theoretical, centered on advanced stochastic calculus, martingale characterizations, and McKean-Vlasov jump-diffusion processes, indicating very high mathematical density. Empirically, it presents a unified algorithm framework but relies on simulated experiments with ‘satisfactory performance’ rather than providing backtest-ready code, datasets, or statistical metrics for real financial applications.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Unified continuous-time Q-learning<br>for MFG & MFC with unknown<br>distributions in jump-diffusion models"]
  B["Core Methodology:<br>Decoupled Integrated Q-function<br>(Decoupled Iq-function)"]
  C["Data/Input:<br>Jump-diffusion model dynamics<br>Representative agent states"]
  D["Computational Process:<br>Unified Q-learning algorithm<br>using test policies &<br>averaged martingale orthogonality"]
  E["Outcome 1:<br>MFG Equilibrium Policy<br>(Decoupled Iq-function characterization)"]
  F["Outcome 2:<br>MFC Optimal Policy<br>(Decoupled Iq-function characterization)"]
  F1["Performance: Satisfactory results in<br>financial applications<br>Exact parameterization obtained"]

  A --> B
  B --> C
  C --> D
  D --> E
  D --> F
  E --> F1
  F --> F1

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper devises a unified continuous-time q‑learning algorithm that works for both mean‑field games and mean‑field control problems by using test policies and an averaged martingale orthogonality condition.

Source sentence
Based on these theoretical findings, we devise a unified q-learning algorithm for both MFG and MFC problems by utilizing test policies and the averaged martingale orthogonality condition.

Assumes 5

  • The decoupled value function Jd belongs to the function space C^{2,2}(R^d × P_2(R^d)) with continuous derivatives ∂xJ, ∂xxJ, ∂µJ and ∂v∂J. model
    Source sentence
    We introduce the space C2,2(Rd × P2(Rd)) the set of functions J : Rd × P2(Rd) such that ∂xJ(x, µ), ∂xxJ(x, µ), ∂µJ(x, µ)(v) and ∂v∂J(x, µ)(v) exist and are jointly continuous.
  • Assumption 2.4: the Shannon entropy of the policy h is well defined and integrable. statistical
    Source sentence
    For h ∈Π, we also denote by Eh(t, x, µ) the Shannon entropy of h, i.e., Eh(t, x, µ) = R A h(a|t, x, µ) log h(a|t, x, µ)da. Assumption 2.4.
  • Assumption 2.5 holds when the model coefficients are linear in the state and mean field, and the policy is Gaussian with smooth mean and variance. model
    Source sentence
    As shown in Frikha et al. (2023), Assumption 2.4 is satisfied if b and σ are linear functions of x and ¯µ := R Rd zµ(dz), r and g are quadratic functions of x and ¯µ := R Rd zµ(dz), and π(a|t, x, µ) is Gaussian with its mean and variance as smooth functions of (t, x, µ).
  • If d=1, the mapping Ψ can be taken as the quantile function of the distribution µ. model
    Source sentence
    As d = 1, the mapping Ψ can be taken as Ψ(e, µ) = Qµ(e), where Qµ is the quantile function of µ.
Show 1 more
  • The jump‑diffusion dynamics follow a controlled Poisson process with intensity η>0 and the jump size is the control variable. model
    Source sentence
    6.2 Non-LQ MFG and MFC examples of jump control In this subsection, we consider the model dynamics as the controlled jump-diffusion process described by dXπ s = Xπ s−(aπ s dNs + σdWs), (6.7) where dNs is a Poisson process with intensity η > 0. We consider the scenario where only the jump size is controlled by the control process aπ t , which has applications in some risk control and risk management problems.

Data

Synthetic data only.

Costs

Not applicable The paper reports no trading or execution result.

Finds 3

  • For several financial applications in the jump‑diffusion setting, the exact parameterization of the decoupled Iq‑functions and value functions is obtained and the q‑learning algorithm shows satisfactory performance. theoretical
    Source sentence
    For several financial applications in the jump-diffusion setting, we obtain the exact parameterization of the decoupled Iq-functions and the value functions, and illustrate our q-learning algorithm with satisfactory performance.
  • The decoupled Iq‑function enables a unified policy evaluation rule for both MFG and MFC problems. theoretical
    Source sentence
    As a result, the decoupled Iq-function and the focus of the representative agent actually enable us to study the q-learning algorithms for both MFG and MFC problems.
  • The mean‑field equilibrium for MFG is characterized as the fixed point of the Gibbs measure of the decoupled Iq‑function, while the mean‑field optimal policy for MFC is obtained via a different use of the same function. theoretical
    Source sentence
    As the first main contribution of the present paper, it is shown that the mean-field equilibrium (MFE for short) of MFG and the mean-field optimal policy of MFC can be related to the decoupled Iq-function, albeit in different ways. Precisely, MFE for the MFG is characterized by the fixed point of the Gibbs measure of the decoupled Iq-function; see the expression in (3.5).

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.