Paper: arXiv 2501.07508

Authors: Haonan Xu, Alessio Brini

Abstract

This paper applies deep reinforcement learning (DRL) to optimize liquidity provisioning in Uniswap v3, a decentralized finance (DeFi) protocol implementing an automated market maker (AMM) model with concentrated liquidity. We model the liquidity provision task as a Markov Decision Process (MDP) and train an active liquidity provider (LP) agent using the Proximal Policy Optimization (PPO) algorithm. The agent dynamically adjusts liquidity positions by using information about price dynamics to balance fee maximization and impermanent loss mitigation. We use a rolling window approach for training and testing, reflecting realistic market conditions and regime shifts. This study compares the data-driven performance of the DRL-based strategy against common heuristics adopted by small retail LP actors that do not systematically modify their liquidity positions. By promoting more efficient liquidity management, this work aims to make DeFi markets more accessible and inclusive for a broader range of participants. Through a data-driven approach to liquidity management, this work seeks to contribute to the ongoing development of more efficient and user-friendly DeFi markets.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts like Markov Decision Processes, Proximal Policy Optimization, and Loss-Versus-Rebalancing modeling, indicating high math complexity. It uses a rolling window approach on real-world market data for training and testing, with a concrete comparison against heuristics, demonstrating high empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Optimize Liquidity Provision in Uniswap v3<br>for Improved DeFi Accessibility"] --> B["Model Formulation:<br>Markov Decision Process (MDP)"]
  B --> C["Core Methodology:<br>Deep Reinforcement Learning (PPO Algorithm)"]
  C --> D["Data Inputs:<br>Historical Price Dynamics<br>Market Regime Shifts"]
  D --> E["Computational Process:<br>Active LP Agent Training via Rolling Window"]
  E --> F["Key Findings:<br>DRL Strategy Outperforms Static Heuristics<br>Fee Maximization & Impermanent Loss Mitigation"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a deep reinforcement learning (DRL) approach using Proximal Policy Optimization to optimize liquidity provisioning in Uniswap v3, enabling an active LP agent to dynamically rebalance its positions.

Source sentence
a deep reinforcement learning (DRL) approach to optimize liquidity provisioning in Uniswap v3, enabling an LP to act as an agent that systematically re‑balances its position over time.

Assumes 6

  • Liquidity provisioning is modeled as a Markov Decision Process. model
    Source sentence
    We model the liquidity provision task as a Markov Decision Process (MDP) and train an active liquidity provider (LP) agent using the Proximal Policy Optimization (PPO) algorithm.
  • Hourly discretization of price data is sufficient for decision making. data
    Source sentence
    At each discrete hourly step, the agent decides whether to withdraw and redeploy liquidity with adjusted price boundaries or to maintain the current interval, balancing risk and profitability.
  • Rolling‑window training captures regime shifts and reflects realistic market conditions. market
    Source sentence
    We choose this rolling window approach to reflect the practical challenges of deploying DRL models in real‑world financial markets, particularly in the volatile cryptocurrency market.
  • Gas fees are a fixed $5 per deployment action. execution
    Source sentence
    where g represents the fixed costs for the gas fees which we set at $5 based on Etherscan’s gas tracker.
Show 2 more
  • Trading fee rate δ is constant for the pool. execution
    Source sentence
    where δ represents the fixed fee rate.
  • Liquidity providers face a penalty proportional to instantaneous LVR. model
    Source sentence
    ... (Milionis et al. 2023) defines instantaneous LVR as ℓt(σ, pt) = Lσ2 /4 √pt

Data

  • Hourly WETH/USDC Uniswap v3 pool data · Uniswap Ethereum subgraph crypto-onchain public
    5 May 2021 1:00 am to 29 January 2024 7: · WETH/USDC pair at 0.05% fee tier
    Source sentence
    The dataset for this study consists of hourly data spanning from 5 May 2021 at 1:00 am, marking the inception of Uniswap v3, to 29 January 2024 at 7:00 pm. We focus on the WETH/USDC Uniswap pool at the 0.05% fee tier.

Costs

Modeled Costs are part of the reported headline results.

Gas fees are modeled as a fixed $5 per position change; trading fees are modeled with a fixed fee rate δ as in equations (5) and (6).

Source sentence
The reward function includes a penalty for gas fees incurred when deploying liquidity positions

Finds 3

  • The active LP agent outperforms heuristic passive LP strategies in 7 out of 11 out‑of‑sample windows, achieving higher cumulative rewards. performance
    Source sentence
    Our empirical results reveal that the active LP out‑performs heuristic‑based passive LP strategies in diverse market conditions, achieving higher cumulative rewards in 7 out of 11 out‑of‑sample windows.
  • The active LP adapts liquidity intervals dynamically, reducing unnecessary rebalancing costs compared to the passive LP. performance
    Source sentence
    to maximize fee collection with maintaining inactive positions during anticipated mean‑reversion periods, reducing unnecessary costs.
  • In stationary price periods the active LP redeploys liquidity only once, while the passive LP’s heuristic rebalancing leads to lower cumulative rewards. performance
    Source sentence
    In this case, the active LP (bottom‑right panel) redeploys liquidity only once, recognizing the stationary nature of the price. Conversely, the passive LP (bottom‑left panel) adheres to its heuristic rebalancing strategy, which results in unnecessary rebalancing and lower cumulative rewards.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 19 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (33)

  1. Risks and returns of uniswap v3 liquidity providers (2022) — link
  2. Uniswap v3 core (2021)
  3. Automated Market Making and Loss-Versus-Rebalancing (2023) — link
  4. Decentralized finance (2020)
  5. The technology of decentralized finance (DeFi) (2024)
  6. Complexity-Approximation Trade-offs in Exchange Mechanisms: AMMs vs. LOBs — → 2302.11652
  7. Impermanent loss in uniswap v3 (2021) — link
  8. Concentrated liquidity in automated market makers (2021)
and 25 more
  1. Cyclic arbitrage in decentralized exchanges (2022)
  2. TA-Lib: Technical Analysis Library in Python
  3. Etherscan Gas Tracker
  4. Proximal policy optimization algorithms (2017) — link
  5. Simulation-based optimization of Markov reward processes (2001) — link
  6. Policy Gradient Methods for Reinforcement Learning with Function Approximation (2000)
  7. Sok: Decentralized finance (defi) (2022)
  8. Sok: Decentralized exchanges (dex) with automated market maker (amm) protocols (2023)
  9. The costs of swapping on the uniswap protocol — → 2309.13648
  10. An empirical study of market inefficiencies in Uniswap and SushiSwap (2022)
  11. Decentralized exchange: The uniswap automated market maker (2021)
  12. Strategic liquidity provision in uniswap v3 (2021) — link
  13. Adaptive Liquidity Provision in Uniswap V3 with Deep Reinforcement Learning (2023) — link
  14. Just-in-time liquidity on the uniswap protocol (2022)
  15. Backtesting Framework for Concentrated Liquidity Market Makers on Uniswap V3 Decentralized Exchange — → 2410.09983
  16. Uniswap: Impermanent loss and risk profile of a liquidity provider (2021) — link
  17. Automated market makers: A practical guide to decentralized exchanges and cryptocurrency trading (2023)
  18. dydx: A standard for decentralized margin trading and derivatives (2018)
  19. The Role of Decentralized Finance (DeFi) in Reshaping Global Financial Inclusion: Opportunities and Risks (2024)
  20. Training language models to follow instructions with human feedback (2022)
  21. Delve into PPO: Implementation Matters for Stable RLHF (2023)
  22. Decentralized Finance and Automated Market Making: Predictable Loss and Optimal Liquidity Provision (2024)
  23. Uniswap liquidity provision: An online learning approach (2023)
  24. Differential Liquidity Provision in Uniswap v3 and Implications for Contract Design (2022)
  25. Liquidity Math in Uniswap v3 (2021)

3 of 33 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.