Paper: arXiv 2603.21089

Authors: Flemming Holtorf, Sungho Shin

Abstract

We present an approximate dynamic programming framework for designing degradation-aware market participation policies for battery energy storage systems. The approach employs a tailored value function approximation that reduces the state space to state of charge and battery health, while performing dynamic programming along a pseudo-time axis encoded by state of health. This formulation enables an offline/online computation split that separates long-term degradation dynamics (months to years) from short-term market dynamics (seconds to minutes) – a timescale mismatch that renders conventional predictive control and dynamic programming approaches computationally intractable. The main computational effort occurs offline, where the value function is approximated via coarse-grained backward induction along the health dimension. Online decisions then reduce to a real-time tractable one-step predictive control problem guided by the precomputed value function. This decoupling allows the integration of high-fidelity physics-informed degradation models without sacrificing real-time feasibility. Backtests on historical market data show that the resulting policy outperforms several benchmark strategies with optimized hyperparameters.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents an innovative Approximate Dynamic Programming (ADP) approach to decouple long-term degradation from short-term market dynamics. The methodology is mathematically sound and supported by empirical backtests using historical market data, showing performance gains over benchmark strategies.

Research Flowchart

  flowchart TD
    A["Research Goal: Resolve timescale mismatch between real-time market trading & long-term battery degradation"] --> B["Inputs: Market Data & Physics-Informed Battery Degradation Models"]
    
    subgraph Methodology ["Methodology: Approximate Dynamic Programming (ADP)"]
        C["State Space Reduction: Reduce states to State of Charge (SoC) & State of Health (SoH)"]
        D["Offline Phase: Coarse-grained Backward Induction along SoH pseudo-time axis"]
        E["Online Phase: Real-time 1-step predictive control guided by precomputed Value Function"]
    end

    B --> C
    C --> D
    D --> E
    E --> F["Outcomes: Outperforms benchmark strategies & achieves real-time computational feasibility"]

Cited References (42)

  1. Declining Costs, Shifting Revenues: Evolving Business Case for Battery Storage — link
  2. Market- Based Energy Management Model of a Building Microgrid Considering Battery Degradation — link
  3. Deep reinforcement learning: A brief survey (2017)
  4. Nonlinear Programming (1999)
  5. Quantifying Electricity Storage Arbitrage Opportunities in Short-Term Electricity Markets in the CWE Region — link
  6. Multiscale model predictive control of battery systems for frequency regulation markets using physics-based models (2020)
  7. Multivariate probabilistic forecasting of intraday electricity prices using normalizing flows (2023)
  8. Feudal reinforcement learning (1992)
and 34 more
  1. The Potential for Battery Energy Storage to Provide Peaking Capacity in the United States — link
  2. Modeling of galvanostatic charge and discharge of the lithium/polymer/insertion cell (1993)
  3. Battery calendar aging and machine learning (2022)
  4. ENTSO‑E Transparency Platform: Central collection and publication of pan‑European electricity market data (2025) — link
  5. Improved Battery Storage Valuation Through Degradation Reduction — link
  6. Genetic identification and fisher identifiability analysis of the Doyle–Fuller–Newman model from experimental cycling of a LiFePO4 cell (2012)
  7. Storage Futures Study : Economic Potential of Diurnal Storage in the U . S . Power Sector — link
  8. Dynamic cycling enhances battery lifetime (2025)
  9. Quantitative validation of calendar aging models for lithium-ion batteries (2018)
  10. Calendar aging of lithium-ion batteries (2016)
  11. Equation-free multiscale computation: Algorithms and applications (2009)
  12. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation (2016)
  13. A decade of insights: Delving into calendar aging trends and implications (2025)
  14. A single particle model with chemical/mechanical degradation physics for lithium ion battery State of Health (SOH) estimation (2018)
  15. Battery-health conscious power management in plug-in hybrid electric vehicles via electrochemical modeling and stochastic control (2012)
  16. Cycle life modeling of lithium-ion batteries (2004)
  17. Lithium-ion battery degradation: how to model it (2022)
  18. Battery Energy Storage Systems in Energy and Reserve Markets — link
  19. Optimal charging of li-ion batteries via a single particle model with electrolyte and thermal dynamics (2017)
  20. Development of first principles capacity fade model for Li-ion cells (2004)
  21. Review and performance comparison of mechanical-chemical degradation models for lithium-ion batteries (2019)
  22. RTE Services Portal: Access to Network Connection, Market Data, APIs and Customer Services (2025) — link
  23. Multiphase porous electrode theory (2017)
  24. Integrated Multiscale Design , Market Participation , and Replacement Strategies for Battery Energy Storage Systems — link
  25. Approximate solutions for galvanostatic discharge of spherical particles I. Constant diffusion coefficient (2001)
  26. A boundary condition for porous electrodes (2004)
  27. Accelerated battery lifetime simulations using adaptive inter-cycle extrapolation algorithm (2021)
  28. Temporal credit assignment in reinforcement learning (1984)
  29. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning (1999)
  30. Feudal networks for hierarchical reinforcement learning (2017)
  31. On the Implementation of an Interior-Point Filter Line-Search Algorithm for Large-Scale Nonlinear Programming — link
  32. Recent progress in reinforcement learning and adaptive dynamic programming for advanced control applications (2023)
  33. Factoring the Cycle Aging Cost of Batteries Participating in Electricity Markets — link
  34. An Age-Dependent Battery Energy Storage Degradation Model for Power System Operations — link

0 of 42 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.