Paper: arXiv 2604.23975

Authors: Ryuji Hashimoto, Ryosuke Takata, Masahiro Suzuki, Yuki Tanaka, Kiyoshi Izumi

Abstract

Agent-based models provide a constructive approach to studying emergent dynamics in life-like systems composed of interacting, adaptive agents. Financial markets serve as a canonical example of such systems, where collective price dynamics arise from individual decision-making. In this modeling tradition, investor behavior has typically been captured by two distinct mechanisms – learning and heterogeneous preferences – which have been explored as separate paradigms in prior studies. However, the impact of their joint modeling on the resulting collective dynamics remains largely unexplored. We develop a multi-agent reinforcement learning framework in which agents endowed with heterogeneous risk aversion, time discounting, and information access learn trading strategies interactively within an artificial market. The experiment reveals that (i) learning under heterogeneous preferences drives agents to develop functionally differentiated strategies through interaction, rather than trait-specific rules, resulting in role specialization, and (ii) the interactions by the differentiated agents are essential for the emergence of realistic market dynamics such as fat-tailed price fluctuations and volatility clustering. Overall, this study demonstrates that the joint design of heterogeneous preferences and learning mechanisms enables the synthesis of an artificial market in which adaptive interactions drive the self-organization of a market ecology, providing a computational realization of the Adaptive Market Hypothesis.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper combines multi-agent reinforcement learning (MARL) formalisms and POMDP structures with an empirical calibration strategy using optimal transport (OT) to match synthetic data to real-world market stylized facts. The model is conceptually sophisticated in its application of complex adaptive systems theory while maintaining a link to empirical validation through market microstructure metrics.

Research Flowchart

  flowchart TD
    A[Research Goal: Joint Impact of Learning & Heterogeneous Preferences] --> B[Methodology: Multi-Agent Reinforcement Learning]
    B --> C[Inputs: Agents with Heterogeneous Risk, Time Discounting, & Information]
    C --> D[Process: Interactive Strategy Learning in Artificial Market]
    D --> E[Outcome 1: Role Specialization/Functional Differentiation]
    D --> F[Outcome 2: Emergent Realistic Market Dynamics/AMH]
    E --> G[Conclusion: Market as Self-Organized Ecology]
    F --> G

Cited References (86)

  1. An agent-based model of financial market efficiency dynamics (2022)
  2. How market ecology explains market malfunction (2021)
  3. Market force, ecology and evolution (2002)
  4. Why are economists sceptical about agent-based simulations? (2005)
  5. Agent-based models: Understanding the economy from the bottom up (2016)
  6. Chapter 24 Agent-based computational finance (2006)
  7. Agent-based modeling in economics and finance: Past, present, and future (2022)
  8. Agent-based computational economics: growing economies from the bottom up (2002)
and 78 more
  1. Deep Reinforcement Learning in Agent Based Financial Market Simulation (2020)
  2. Reinforcement learning in agent-based market simulation: Unveiling realistic stylized facts and behavior (2024)
  3. Modeling the stylized facts in finance through simple nonlinear adaptive systems (2002)
  4. Impact of high-frequency trading with an order book imbalance strategy on agent-based stock markets (2023)
  5. “Artificial Societies” and the social sciences (2002)
  6. What is artificial life today, and where should it go? (2024)
  7. The adaptive markets hypothesis: Market efficiency from an evolutionary perspective (2004)
  8. Reconciling efficient markets with behavioral finance: The adaptive markets hypothesis (2004)
  9. Agent-based computational models and generative social science (1999)
  10. Empirical validation of agent-based models: Alternatives and prospects (2007)
  11. Collective intelligence: A unifying concept for integrating biology across scales and substrates (2024)
  12. Complex adaptive systems: An introduction to computational models of social life (2007)
  13. Complexity and the economy (1999)
  14. The distribution of risk aversion (2007)
  15. Impact of investor’s varying risk aversion on the dynamics of asset price fluctuations (2006)
  16. The economy needs agent-based modeling (2009)
  17. High-dimensional continuous control using generalized advantage estimation (2018) — link
  18. Modeling bounded rationality in multi-agent simulations using rationally inattentive reinforcement learning (2022)
  19. Phantom - A RL-driven multi-agent framework to model complex systems (2023)
  20. Learning and calibrating heterogeneous bounded rational market behaviour with multi-agent reinforcement learning (2024)
  21. CoFinDiff: Controllable financial diffusion model for time series generation (2025)
  22. Complementarity in biological systems: A complexity view (2013)
  23. Institutional investors and stock market volatility (2006)
  24. Denoising diffusion probabilistic models (2020)
  25. Dynamic programming algorithm optimization for spoken word recognition (1978)
  26. Are stock markets really efficient? Evidence of the adaptive market hypothesis (2016)
  27. Exchange-rate return predictability and the adaptive markets hypothesis: Evidence from major foreign exchange rates (2012)
  28. Scaling in stock market data: Stable laws and beyond (1997)
  29. The limiting extremal behaviour of speculative returns: an analysis of intra-daily data from the Frankfurt Stock Exchange (1998)
  30. The virtues and vices of equilibrium and the future of financial economics (2009)
  31. Asset prices with heterogeneity in preferences and beliefs (2013)
  32. Investor heterogeneity, asset pricing and volatility dynamics (2009)
  33. A simulation analysis of the microstructure of double auction markets (2002)
  34. The impact of heterogeneous trading rules on the limit order book and order flows (2009)
  35. Financial power laws: Empirical evidence, models, and mechanisms (2016)
  36. Generative adversarial networks (2014) — link
  37. Least squares generative adversarial networks (2017)
  38. Generalized autoregressive conditional heteroskedasticity (1986)
  39. Volatility persistence and stock valuations: Some empirical evidence using GARCH (1988)
  40. Modelling bounded rationality in multi-agent interactions by generalized recursive reasoning (2021)
  41. Historical Data (2025)
  42. The effects of payoff magnitude and heterogeneity on behavior in 2$\times$2 games with unique mixed strategy equilibria (2000)
  43. Evolution and time horizons in an agent-based stock market (1999)
  44. Heterogeneous agent models in financial markets: A nonlinear dynamics approach (2019)
  45. Heterogeneity and self-organization of complex systems through an application to financial market with multiagent systems (2017)
  46. Heterogeneous risk preferences in financial markets (2017)
  47. A simple general approach to inference about the tail of a distribution (1975)
  48. An experimental test of preferences for the distribution of income and individual risk aversion (1994)
  49. From KISS to KIDS – An `Anti-simplistic’ modelling approach (2005)
  50. The variation of certain speculative prices (1963)
  51. The behavior of stock-market prices (1965)
  52. Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria (1998)
  53. Heterogeneous beliefs and routes to chaos in a simple asset pricing model (1998)
  54. Financial markets as nonlinear adaptive evolutionary systems (2001)
  55. Artificial economic life: A simple model of a stockmarket (1994)
  56. Heterogeneous beliefs, risk and learning in a simple asset pricing model (2002)
  57. Rational heuristics? Expectations and behaviors in evolving economies with heterogeneous interacting agents (2020)
  58. Towards multi-agent reinforcement learning-driven over-the-counter market simulations (2024)
  59. Between individual brains and collective behavior: Multi-level emergence in a group formation task (2023)
  60. Computational optimal transport: With applications to data science (2019)
  61. Interpretable market simulations via optimal transport: Power law decomposition and implications for market design (2025)
  62. PAMS: Platform for artificial market simulations (2023) — link
  63. POT: Python optimal transport (2021)
  64. Proximal policy optimization algorithms (2017) — link
  65. What matters in on-policy reinforcement learning? A large-scale empirical study (2020) — link
  66. Implications of rational inattention (2003)
  67. Get real: Realism metrics for robust limit order book market simulations (2021)
  68. Reinforcement learning: An introduction (2018)
  69. Soft Actor-Critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
  70. Calibration of shared equilibria in general sum partially observable Markov games (2020)
  71. How to keep it adequate: A protocol for ensuring validity in agent-based simulation (2023)
  72. Empirical properties of asset returns: stylized facts and statistical issues (2001)
  73. Generating synthetic data in finance: Opportunities, challenges and pitfalls (2021)
  74. Time varying risk aversion (2018)
  75. The relation between price changes and trading volume: A survey (1987)
  76. An analysis of U.S. stock price behavior using wavelets (1995)
  77. Learning to simulate realistic limit order book markets from data as a world agent (2022)
  78. An agent-based modeling approach to study price impact (2012)

0 of 86 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.