{"paper_id":"2512.17929","title":"Reinforcement Learning for Monetary Policy Under Macroeconomic Uncertainty: Analyzing Tabular and Function Approximation Methods","authors":["Tony Wang","Kyle Feinstein","Sheryl Chen"],"paper_date":"2025-12-09","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2512.17929v2","page_url":"https://thequant.space/flowcharts/reinforcement-learning-for-monetary-policy-under-macroeconom/","scores":{"empirical_rigor":7.2,"math_complexity":6.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper implements and compares nine reinforcement‑learning approaches on a data‑driven monetary‑policy environment constructed from historical macroeconomic data, ranging from tabular methods to deep and Bayesian techniques.","quote":"This paper makes several contributions to the intersection of RL and monetary policy: (1) We implement and compare nine different RL approaches on a data-driven monetary policy environ-ment, ranging from tabular methods to deep learning and Bayesian approaches.","summary":"We study how a central bank should dynamically set short-term nominal interest rates to stabilize inflation and unemployment when macroeconomic relationships are uncertain and time-varying. We model monetary policy as a sequential decision-making problem where the central bank observes…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The economy dynamics are modeled as a linear‑Gaussian transition process.","type":"model","stated":"explicit","quote":"Using publicly accessible historical Federal Reserve Economic Data (FRED), we construct a linear-Gaussian transition model and implement a discrete-action Markov Decision Process with a quadratic loss reward function.","support":"full"},{"assumption":"The central bank observes macroeconomic conditions quarterly and selects from a discrete set of interest‑rate adjustments.","type":"data","stated":"explicit","quote":"We model monetary policy as a sequential decision-making problem where the central bank observes macroeconomic conditions quarterly and chooses interest rate adjustments. We chose to compare nine different reinforcement learning style approaches against Taylor Rule and naive baselines, including tabular Q-learning variants, SARSA, Actor-Critic, Deep Q-Networks, Bayesian Q-learning with uncertainty quantification,","support":"full"},{"assumption":"The transition model is fixed and estimated from historical data, not updated during agent interaction.","type":"model","stated":"explicit","quote":"One central limitation is that agents interact with a fixed transition model estimated from historical data, not the real macroeconomic system.","support":"full"},{"assumption":"The reward function is a quadratic loss based on inflation, unemployment and smoothing, with weight choices that are somewhat arbitrary.","type":"statistical","stated":"explicit","quote":"The reward function, while based on established dual mandate principles, involves somewhat arbitrary weight choices.","support":"full"},{"assumption":"The discrete action space limits policy flexibility compared to continuous rate setting.","type":"execution","stated":"explicit","quote":"The discrete action space 5 Figure 2: Learning dynamics dashboard showing training curves, epsilon decay, loss evolution, uncertainty evolution, convergence analysis, training stability, performance distributions, and sample efficiency across all methods. limits policy flexibility compared to continuous interest rate setting.","support":"full"},{"assumption":"Episodes are limited to 80 quarters, which may not capture long‑term cycles.","type":"execution","stated":"explicit","quote":"Our 80-quarter horizon may not adequately capture long-term economic cycles and structural changes.","support":"full"}]},"datasets":{"status":"available","items":[{"name":"Federal Reserve Economic Data (FRED)","provider":"Federal Reserve","kind":"macro","access":"public","period":"1955-2025","universe":null,"quote":"Using publicly accessible historical Federal Reserve Economic Data (FRED), we construct a linear-Gaussian transition model and implement a discrete-action Markov Decision Process with a quadratic loss reward function. 3 Methodology 3.1 Environment and Data We construct our state space from quarterly U.S. macroeconomic data obtained from the Federal Reserve Economic Data (FRED) API [3], spanning 1955-2025.","support":"full","known_id":"fred"}],"known":[{"id":"fred","name":"FRED","provider":"Federal Reserve Bank of St. Louis","kind":"macro","access":"public","quote":"3 Methodology 3.1 Environment and Data We construct our state space from quarterly U.S. macroeconomic data obtained from the Federal Reserve Economic Data (FRED) API [3], spanning 1955-2025.","mentions":8,"used_in":3}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"weekly-monthly"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R3","candidates":{"front_matter":"n/a","evidence_card":"n/a","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"unclear","findings":[{"finding":"Standard tabular Q‑learning (legacy) achieved the best mean return of –615.13 ± 309.58, outperforming all enhanced RL methods and traditional baselines.","kind":"performance","quote":"Notably, despite its simplicity, standard tabular Q-learning achieved the best performance (-615.13 +- 309.58 mean return), outperforming both enhanced RL methods and traditional policy rules. Contrary to our original expectations, standard tabular Q-learning achieved the best performance with mean return −615.13 ± 309.58, significantly outperforming both enhanced RL methods and traditional baselines.","support":"full"},{"finding":"The performance gap between the best method (Q‑learning legacy) and the worst (SARSA) was 171.10 return units, a 21.8% relative improvement.","kind":"performance","quote":"The performance gap between the best (Q-learning legacy) and worst (SARSA) methods is 171.10 return units, representing a 21.8% relative improvement.","support":"full"},{"finding":"Simpler RL methods can outperform both sophisticated RL techniques and traditional policy rules in the monetary‑policy domain.","kind":"performance","quote":"(3) We demonstrate that simpler RL methods can outperform both sophisticated RL techniques and traditional policy rules in this domain. Contrary to our original expectations, standard tabular Q-learning achieved the best performance with mean return −615.13 ± 309.58, significantly outperforming both enhanced RL methods and traditional baselines.","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"none","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":12,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T19:45:10Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}