Paper: arXiv 2402.10803
Authors: Johann Lussange, Stefano Vrizzi, Stefano Palminteri, Boris Gutkin
Abstract
Building on a previous foundation work (Lussange et al. 2020), this study introduces a multi-agent reinforcement learning (MARL) model simulating crypto markets, which is calibrated to the Binance’s daily closing prices of \(153\) cryptocurrencies that were continuously traded between 2018 and 2022. Unlike previous agent-based models (ABM) or multi-agent systems (MAS) which relied on zero-intelligence agents or single autonomous agent methodologies, our approach relies on endowing agents with reinforcement learning (RL) techniques in order to model crypto markets. This integration is designed to emulate, with a bottom-up approach to complexity inference, both individual and collective agents, ensuring robustness in the recent volatile conditions of such markets and during the COVID-19 era. A key feature of our model also lies in the fact that its autonomous agents perform asset price valuation based on two sources of information: the market prices themselves, and the approximation of the crypto assets fundamental values beyond what those market prices are. Our MAS calibration against real market data allows for an accurate emulation of crypto markets microstructure and probing key market behaviors, in both the bearish and bullish regimes of that particular time period.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced multi-agent reinforcement learning (MARL) and complex agent-based modeling to simulate crypto market microstructure, indicating high mathematical density. It is empirically rigorous, being directly calibrated on real-world Binance data for 153 cryptocurrencies over a 5-year period (2018-2022), covering both bearish and bullish regimes.
Research Flowchart
flowchart TD
A["Research Goal<br>Model crypto markets with<br>Multi-Agent Reinforcement Learning"] --> B{"Key Methodology"}
B --> B1["Data: 153 Cryptocurrencies<br>Binance Daily Closing Prices<br>2018-2022"]
B --> B2["Agents: RL-equipped<br>vs. Zero-Intelligence/Single Agent"]
B --> B3["Information Sources<br>Market Prices & Fundamental Values"]
B --> B4["Calibration<br>Emulate Market Microstructure"]
B --> B5["Regime Analysis<br>Bearish vs. Bullish<br>COVID-19 Era"]
B --> C["Computational Process"]
C --> C1["Bottom-up Complexity Inference"]
C --> C2["Multi-Agent Simulation<br>Interaction of Autonomous Agents"]
C --> C3["Robustness Check<br>Market Volatility"]
C --> D{"Outcomes"}
D --> D1["Accurate Emulation of Crypto Markets"]
D --> D2["Insights into Market Microstructure"]
D --> D3["Behavioral Probing across Regimes"]
D --> D4["Validation of RL approach<br>vs. Traditional ABM/MAS"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a multi‑agent reinforcement learning (MARL) model called SYMBA to simulate crypto markets using calibrated agents.
Source sentence
this study introduces a multi-agent reinforcement learning (MARL) model simulating crypto markets, which is calibrated to the Binance's daily closing prices of 153 cryptocurrencies that were continuously traded between 2018 and 2022.
Assumes 6
- The behavior of simulated agents accurately reflects that of real‑world investors. model
Source sentence
i- that the behavior of the simulated agents accurately reflects that of real-world investors
- Transaction limit orders in the simulated order book faithfully represent actual crypto market order dynamics. market
Source sentence
ii- that the transaction limit orders simulated in the order book faithfully represent the dynamics and characteristics of actual crypto market orders
- Agents can only buy, sell, or hold assets (long‑only strategy). model
Source sentence
our approach simplifies the interaction of any agent, regardless of its behavior or strategy, into three distinct possibilities: buying, selling, or holding assets (a long-only strategy)
- Network fees, risk‑free rate and APR are uniform across all simulated crypto assets. execution
Source sentence
we assume these financial parameters to be uniform across simulated crypto assets, with network fees set at b = 0.1%, the risk‑free rate at R = 1%, and the APR at D = 3%
Show 2 more
- Each time step corresponds to a single trading day (24/7 crypto market). data
Source sentence
each time step corresponds to a single trading day, or actually a trading day and night, since contrary to stock markets, crypto markets like Binance trade 24/7
- The model does not include short‑selling, leveraging, derivatives, meta‑orders, market orders, intraday effects, or regulatory constraints. other
Source sentence
the lack of diverse trading strategies available on Binance, such as short‑selling, leveraging, derivatives, metaorders, market orders, and more. iv- Neglecting intraday and seasonal market effects. v- Disregarding legal and regulatory constraints
Data
- CryptoTick daily closing prices and volumes market-prices public
Source sentence
we employed high-quality, industry-standard daily closing prices and trading volumes from CryptoTick, for 153 crypto assets listed on Binance. This dataset covers the period from September 27th, 2018, to September 27th, 2022
Costs
Unclear - sources disagree
Source sentence (the authors' words)
we assume these financial parameters to be uniform across simulated crypto assets, with network fees set at b = 0.1%, the risk‑free rate at R = 1%, and the APR at D = 3%
Finds 5
- The calibrated SYMBA model reproduces key crypto market microstructure features such as non‑normal returns, volume clustering, and decaying price log‑return autocorrelations. statistical
Source sentence
Our results offer valuable insights into the complexity and behavior of crypto markets, highlighting the model’s ability to emulate key market characteristics such as non‑normal returns, volume clustering, and decaying price log‑returns autocorrelations.
- Simulated logarithmic price return distributions closely match those of real Binance data. pricing-calibration
Source sentence
Figure 7: Comparative Distribution of Logarithmic Price Returns: The dashed black curve represents real data, while the continuous red curve represents simulated data.
- Increasing the number of agents reduces short‑term price fluctuations linearly. statistical
Source sentence
augmenting the count of agents (I) displayed a linear correlation with the reduction of short-term price fluctuations.
- Larger gesture scalar and cointegration accuracy lead to linear increases in absolute daily price returns. statistical
Source sentence
increasing the values of the gesture scalar (ζi) and the cointegration accuracy (ν) exhibited a linear escalation in absolute daily price returns.
- Drawdown thresholds above 30% have minimal impact on agent survivability rates. statistical
Source sentence
large drawdown thresholds (L > 30%) showed minimal impact on agent survivability rates.
Checked by others
Supported by
- RIVCoin: an alternative, integrated, CeFi/DeFi-Vaulted Cryptocurrency
Compare the findings
This paper: Increasing the number of agents reduces short‑term price fluctuations linearly.
That paper: Increasing the number of market participants lowers price volatility and improves informational efficiency.
- Is an investor stolen their profits by mimic investors? Investigated by an agent-based model
Compare the findings
This paper: Increasing the number of agents reduces short‑term price fluctuations linearly.
That paper: Increasing the number of additional fundamental agents (AFAs) makes market prices more stable, which leads to a decrease in their profits.
Contradicted by
- Is an investor stolen their profits by mimic investors? Investigated by an agent-based model
Compare the findings
This paper: Increasing the number of agents reduces short‑term price fluctuations linearly.
That paper: Increasing the number of additional technical agents (ATAs) makes market prices more unstable, which leads to an increase in their profits.
- Modelling Opaque Bilateral Market Dynamics in Financial Trading: Insights from a Multi-Agent Simulation Study
Compare the findings
This paper: Increasing the number of agents reduces short‑term price fluctuations linearly.
That paper: Agent diversity (heterogeneous features) promotes market stability, whereas increasing the number of homogeneous agents leads to rapid instability, collapsing by the 25th time step.
1 paper in the archive : JaxMARL-HFT: GPU-Accelerated Large-Scale Multi-Agent …
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (177)
- Security tokens: architecture, smart contract applications and illustrations using SAFE (2020)
- Market microstructure and order book dynamics in cryptocurrency exchanges (2018)
- Order flow analysis of cryptocurrency markets (2019)
- Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia (2023) — link
- Using an artificial financial market for studying a cryptocurrency market (2017)
- Highly accurate protein structure prediction with AlphaFold (2021)
- Toward Reverse Engineering to Economic Analysis: An Overview of Tools and Methodology (2022)
- Reverse engineering financial markets with majority and minority games using genetic algorithms (2013)
and 169 more
- Herding and anchoring in cryptocurrency markets: Investor reaction to fear and uncertainty (2020)
- Dynamic principal market determination: Fair value measurement of cryptocurrency (2023)
- Investors’ beliefs and cryptocurrency prices (2024)
- Approaches to cryptocurrency valuation (2023)
- A survey of blockchain consensus protocols (2023)
- A survey of Layer-two blockchain protocols (2023)
- Bitcoin (2009)
- Distributed ledger technology (2020)
- Tokenomics: Dynamic adoption and valuation (2021)
- Tokenomics: when tokens beat equity (2023)
- A survey of distributed consensus protocols for blockchain networks (2020)
- Multiscale characteristics of the emerging global cryptocurrency market (2021)
- What keeps stablecoins stable? (2023)
- Fundamental pricing of utility tokens (2021)
- Forecasting volatility in bitcoin market (2020)
- Cryptocurrency market contagion: Market uncertainty, market complexity, and dynamic portfolios (2019)
- SYMBA code repository — link
- SYMBA code repository — link
- https://www.ig.com/en/cfd-trading/charges-and-margins
- The Economic Theory of Agency: The Principal’s Problem (1973)
- Discrimination power of measures for nonlinearity in a time series (1997)
- Highly comparative feature-based time-series classification.
- An Agent-Based Model to Explain the Emergence of Stylised Facts in Log Returns
- Reinforcement Learning for Market Making in a Multi-agent Dealer Market
- Analyzing order flows in limit order books with ratios of Cox-type intensities (2019)
- The Impact of Heterogeneous Trading Rules on the Limit Order Book and Order Flows
- Structural stochastic volatility in asset pricing dynamics: Estimation and model contest
- Private information in futures markets: An experimental study
- The relationship between stock market volatility and trading volume: evidence from South Africa
- Order book modeling and financial stability
- Order book microstructure and policies for financial stability
- Learning to forecast, risk aversion, and microstructural aspects of financial stability
- Universal features of price formation in financial markets: perspectives from deep learning
- Deep Reinforcement Learning for Optimizing Portfolio Management
- Enhancing Q-learning for optimal asset allocation
- Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
- Time series feature extraction on basis of scalable hypothesis tests, tsfresh–a python package (2018)
- Scikit-learn, Machine learning in Python (2011)
- Umap: Uniform manifold approximation and projection for dimension reduction (2018) — link
- Generative adversarial nets (2014)
- Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing
- Behavioural and neural characterization of optimistic reinforcement learning
- Market making via reinforcement learning
- Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality
- What is a Free Lunch?
- A Drunk and Her Dog: An Illustration of Cointegration and Error Correction
- A survey of actor-critic reinforcement learning: standard and natural policy gradients
- Human-in-the-loop reinforcement learning
- RL-squared: Fast Reinforcement Learning via Slow Reinforcement Learning
- Prefrontal Cortex as a Meta-Reinforcement Learning System
- Autonomous Helicopter Aerobatics through Apprenticeship Learning
- Theory and application to reward shaping
- Robust Adversarial Reinforcement Learning
- Learning in POMDPs with Monte Carlo tree search
- A bayesian approach for learning and planning in partially observable Markov decision processes
- A Deep Hierarchical Approach to Lifelong Learning in Minecraft
- Asynchronous Methods for Deep Reinforcement Learning
- Deterministic Policy Gradient Algorithms
- A fractional reaction–diffusion description of supply and demand
- More Than the Sum of Its Parts: A Role for the Hippocampus in Configural Reinforcement Learning
- The successor representation in human reinforcement learning
- Strategic Agent-Based Modeling of Financial Markets
- Imaging-based parcellations of the human brain
- A general reinforcement learning algorithm that masters chess, shogi and Go through self-play (2018)
- A detailed heterogeneous agent model for a single asset financial market with trading via an order book
- Policy Analysis Using DSGE Models: An Introduction
- Experience-weighted Attraction Learning in Normal Form Games
- Model-based choices involve prospective neural activity (2015)
- Expectations, Learning, and Business Cycle Fluctuations (2011)
- Heterogeneous expectations in monetary DSGE models (2013)
- Agent-Based Stock Market Model with Endogenous Agents’ Impact
- Empirical properties of asset returns: stylized facts and statistical issues
- Efficient capital markets: A review of theory and empirical work
- The Variation of Certain Speculative Prices
- More stylized facts of financial markets: Leverage effect and downside correlations
- Scaling of the distribution of fluctuations of financial market indices
- Investment rules, margin and market volatility
- Power laws are logarithmic Boltzmann laws
- A microscopic model of the stock market: cycles, booms, and crashes
- Microscopic simulation of the stock market: the effect of microscopic diversity
- Dynamical explanation for the emergence of power law in a stock market model
- Microscopic simulation of financial markets: from investor behavior to market phenomena
- The complex dynamics of a simple stock market model
- New evidence for the power-law distribution of wealth
- Herd behavior and aggregate fluctuations in financial markets
- Social percolation models
- Scaling and criticality in a stochastic multi-agent model of a financial market
- Volatility clustering in financial markets: a microsimulation of interacting agents
- Modelling an imperfect market
- Self-organization of value and demand
- Dynamics of money
- Money and Goldstone modes
- Power, Lévy, exponential and Gaussian-like regimes in autocatalytic financial systems
- Statistical mechanics of socio-economic systems with heterogeneous agents
- The econometrics of financial markets
- Levy-Stable Distributions Revisited: Tail index > 2 does not exclude the Levy-Stable regime
- Size matters: some stylized facts of the stock market revisited
- A long memory property of stock market returns and a new model
- Real and Spurious Long-Memory Properties of Stock-Market Data
- Coherent and random sequences in financial fluctuations
- Tipping points in macroeconomic agent-based models
- A practical, universal, information criterion over nth order markov processes
- Multi-agent-based Order Book Model of financial markets
- Latency arbitrage, market fragmentation, and efficiency: a two-market model
- Chapter 7 - Agent-Based Models for Market Impact and Volatility
- Simulating and analyzing order book data: the queue-reactive model
- Mastering the Game of Go With Deep Neural Networks and Tree Search
- Q-learning
- Statistical models for earthquake occurrences and residual analysis for point processes (1988)
- Autoregressive conditional duration: a new model for irregularly spaced transaction data (1998)
- Measuring the information content of stock trades (1991)
- Modelling financial high frequency data using point processes (2009)
- An introduction to the theory of point processes: volume II: general theory and structure (2007)
- Modelling security market events in continuous time: Intensity based, multivariate point process models (2007)
- Spectra of some self-exciting and mutually exciting point processes (1971)
- Point spectra of some mutually exciting point processes (1971)
- Affine point processes and portfolio credit risk (2010)
- Adaptive estimation for Hawkes processes; application to genome analysis (2010)
- First-and Second-Order Statistics Characterization of Hawkes Processes and Non-Parametric Estimation (2016)
- Quantifying reflexivity in financial markets: Toward a prediction of flash crashes (2012)
- Critical reflexivity in financial markets: a Hawkes process analysis (2013)
- Modelling microstructure noise with mutually exciting point processes (2013)
- Hawkes model for price and trades high-frequency dynamics (2014)
- Estimation of slowly decreasing Hawkes kernels: Application to high frequency order book modelling (2014) — link
- Modelling bid and ask prices using constrained Hawkes processes: Ergodicity and scaling limit (2014)
- Current dividend impacts of FTSE -250 stocks — link
- IG fees of C ontracts F or D ifference — link
- UK one-year gilt reference prices — link
- Limit theorems for nearly unstable Hawkes processes (2015)
- Clustering of order arrivals, price impact and trade path optimisation (2006)
- Modelling trades-through in a limited order book using hawkes processes (2011)
- “Market making” in an order book model and its impact on the spread (2011)
- Modeling financial contagion using mutually exciting jump processes (2010)
- Modeling foreign exchange market activity around macroeconomic news: Hawkes-process approach (2015) — link
- Seismicity analysis through point-process modeling: A review (1999)
- Subcritical and supercritical regimes in epidemic models of earthquake aftershocks (2002)
- Inference of functional connectivity in neurosciences via Hawkes processes (2013)
- Goodness-of-fit tests and nonparametric adaptive estimation for spike train analysis (2014)
- A nonparametric EM algorithm for multiscale Hawkes processes (2011)
- Statistical and stochastic modeling of gang rivalries in Los Angeles (2010)
- Self-exciting point process models of civilian deaths in Iraq (2012)
- Self-exciting point process modeling of crime (2012)
- Robust dynamic classes revealed by measuring the response function of a social system (2008)
- Mixture of Mutually Exciting Processes for Viral Diffusion. (2013)
- Maximum likelihood estimation of Hawkes’ self-exciting point processes (1979)
- Extending earthquakes’ reach through cascading (2008)
- Non-parametric kernel estimation for symmetric Hawkes processes. Application to high frequency financial data (2012)
- Multivariate Hawkes processes: an application to financial data (2011)
- Maximum likelihood from incomplete data via the EM algorithm (1977)
- Hawkes process: Fast calibration, application to trade clustering, and diffusive limit (2014)
- An estimation procedure for the Hawkes process (2015) — link
- Simulation of nonhomogeneous Poisson processes by thinning (1979)
- On Lewis’ simulation method for point processes (1981)
- Perfect simulation of Hawkes processes (2005)
- High-frequency financial data modeling using Hawkes processes (2012)
- Hawkes processes in finance (2015)
- Hawkes branching point processes without ancestors (2001) — link
- The general Theory of Employment, Interest and Money (1936)
- The New Paradigm for Financial Markets: The Credit Crisis of 2008 and What it Means (2008)
- Hedge funds and stock price formation (2018)
- Testing the causality of Hawkes processes with time reversal (2018)
- Building an artificial stock market populated by reinforcement?learning agents (2009) — link
- Coordination through social learning in a general equilibrium model (2017) — link
- Reinforcement Learning, second edition: An Introduction (2018)
- Homeostatic Reinforcement Learning for Integrating Reward Collection and Physiological Stability (2014)
- A Reinforcement Learning Theory for Homeostatic Regulation (2011)
- Actor-critic algorithms for hierarchical decision processes (2006)
- Policy gradient methods for reinforcement learning with function approximation (2000)
- Contextual modulation of value signals in reward and punishment learning (2015)
0 of 177 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.