Paper: arXiv 2512.17929

Authors: Tony Wang, Kyle Feinstein, Sheryl Chen

Abstract

We study how a central bank should dynamically set short-term nominal interest rates to stabilize inflation and unemployment when macroeconomic relationships are uncertain and time-varying. We model monetary policy as a sequential decision-making problem where the central bank observes macroeconomic conditions quarterly and chooses interest rate adjustments. Using publicly accessible historical Federal Reserve Economic Data (FRED), we construct a linear-Gaussian transition model and implement a discrete-action Markov Decision Process with a quadratic loss reward function. We chose to compare nine different reinforcement learning style approaches against Taylor Rule and naive baselines, including tabular Q-learning variants, SARSA, Actor-Critic, Deep Q-Networks, Bayesian Q-learning with uncertainty quantification, and POMDP formulations with partial observability. Notably, despite its simplicity, standard tabular Q-learning achieved the best performance (-615.13 +- 309.58 mean return), outperforming both enhanced RL methods and traditional policy rules. Our results suggest that while sophisticated RL techniques show promise for monetary policy applications, simpler approaches may be more robust in this domain, highlighting important challenges in applying modern RL to macroeconomic policy.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 7.2/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces advanced reinforcement learning concepts and formal MDP/POMDP models, indicating moderate-to-high mathematical density. However, it is backed by concrete empirical implementation using real-world FRED data, specific backtesting procedures, and detailed performance metrics with statistical analysis.

Research Flowchart

  flowchart TD
  A["Research Goal: RL for Monetary Policy under Uncertainty"] --> B["Data & Inputs<br>FRED Macroeconomic Data"]
  B --> C["Model: MDP &<br>Linear-Gaussian Transition"]
  C --> D{"RL Methods Comparison"}
  D --> E["Tabular Methods<br>Q-Learning, SARSA"]
  D --> F["Deep Methods<br>DQN, Actor-Critic"]
  D --> G["Uncertainty Methods<br>Bayesian Q-Learning, POMDP"]
  E & F & G --> H["Key Finding: Simplicity Wins<br>Tabular Q-Learning (-615.13) Outperforms Complex Methods"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper implements and compares nine reinforcement‑learning approaches on a data‑driven monetary‑policy environment constructed from historical macroeconomic data, ranging from tabular methods to deep and Bayesian techniques.

Source sentence
This paper makes several contributions to the intersection of RL and monetary policy: (1) We implement and compare nine different RL approaches on a data-driven monetary policy environ-ment, ranging from tabular methods to deep learning and Bayesian approaches.

Assumes 6

  • The economy dynamics are modeled as a linear‑Gaussian transition process. model
    Source sentence
    Using publicly accessible historical Federal Reserve Economic Data (FRED), we construct a linear-Gaussian transition model and implement a discrete-action Markov Decision Process with a quadratic loss reward function.
  • The central bank observes macroeconomic conditions quarterly and selects from a discrete set of interest‑rate adjustments. data
    Source sentence
    We model monetary policy as a sequential decision-making problem where the central bank observes macroeconomic conditions quarterly and chooses interest rate adjustments. We chose to compare nine different reinforcement learning style approaches against Taylor Rule and naive baselines, including tabular Q-learning variants, SARSA, Actor-Critic, Deep Q-Networks, Bayesian Q-learning with uncertainty quantification,
  • The transition model is fixed and estimated from historical data, not updated during agent interaction. model
    Source sentence
    One central limitation is that agents interact with a fixed transition model estimated from historical data, not the real macroeconomic system.
  • The reward function is a quadratic loss based on inflation, unemployment and smoothing, with weight choices that are somewhat arbitrary. statistical
    Source sentence
    The reward function, while based on established dual mandate principles, involves somewhat arbitrary weight choices.
Show 2 more
  • The discrete action space limits policy flexibility compared to continuous rate setting. execution
    Source sentence
    The discrete action space 5 Figure 2: Learning dynamics dashboard showing training curves, epsilon decay, loss evolution, uncertainty evolution, convergence analysis, training stability, performance distributions, and sample efficiency across all methods. limits policy flexibility compared to continuous interest rate setting.
  • Episodes are limited to 80 quarters, which may not capture long‑term cycles. execution
    Source sentence
    Our 80-quarter horizon may not adequately capture long-term economic cycles and structural changes.

Data

  • Federal Reserve Economic Data (FRED) · Federal Reserve macro public
    1955-2025
    Source sentence
    Using publicly accessible historical Federal Reserve Economic Data (FRED), we construct a linear-Gaussian transition model and implement a discrete-action Markov Decision Process with a quadratic loss reward function. 3 Methodology 3.1 Environment and Data We construct our state space from quarterly U.S. macroeconomic data obtained from the Federal Reserve Economic Data (FRED) API [3], spanning 1955-2025.

Recognised sources: FRED

Costs

Not applicable The paper reports no trading or execution result.

Finds 3

  • Standard tabular Q‑learning (legacy) achieved the best mean return of –615.13 ± 309.58, outperforming all enhanced RL methods and traditional baselines. performance
    Source sentence
    Notably, despite its simplicity, standard tabular Q-learning achieved the best performance (-615.13 +- 309.58 mean return), outperforming both enhanced RL methods and traditional policy rules. Contrary to our original expectations, standard tabular Q-learning achieved the best performance with mean return −615.13 ± 309.58, significantly outperforming both enhanced RL methods and traditional baselines.
  • The performance gap between the best method (Q‑learning legacy) and the worst (SARSA) was 171.10 return units, a 21.8% relative improvement. performance
    Source sentence
    The performance gap between the best (Q-learning legacy) and worst (SARSA) methods is 171.10 return units, representing a 21.8% relative improvement.
  • Simpler RL methods can outperform both sophisticated RL techniques and traditional policy rules in the monetary‑policy domain. performance
    Source sentence
    (3) We demonstrate that simpler RL methods can outperform both sophisticated RL techniques and traditional policy rules in this domain. Contrary to our original expectations, standard tabular Q-learning achieved the best performance with mean return −615.13 ± 309.58, significantly outperforming both enhanced RL methods and traditional baselines.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 12 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (16)

  1. The Courage to Act: A Memoir of a Crisis and Its Aftermath (2015)
  2. Monetary Policy Communications and Their Effects on Household Inflation Expectations (2020)
  3. Bayesian Reinforcement Learning: A Survey (2015)
  4. Planning and Acting in Partially Observable Stochastic Domains (1998)
  5. Actor-Critic Algorithms (2000)
  6. Human-Level Control through Deep Reinforcement Learning (2015)
  7. Identification in Macroeconomics (2018)
  8. Deep Exploration via Bootstrapped DQN (2016)
and 8 more
  1. Slack and Cyclically Sensitive Inflation (2019)
  2. Reinforcement Learning: An Introduction (2018)
  3. Discretion versus Policy Rules in Practice (1993)
  4. Central Bank Models: Lessons from the Past and Ideas for the Future (2016)
  5. Q-Learning (1992)
  6. The Goals of Monetary Policy and How We Pursue Them (2017)
  7. Phillips Curve Instability and Optimal Monetary Policy (2007 (rev.\ November 2015)) — link
  8. FRED API — link

0 of 16 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.