Paper: arXiv 2307.04754

Authors: Francesco Cordoni, Alessio Sancetta

Abstract

A model among many may only be best under certain states of the world. Switching from a model to another can also be costly. Finding a procedure to dynamically choose a model in these circumstances requires to solve a complex estimation procedure and a dynamic programming problem. A Reinforcement learning algorithm is used to approximate and estimate from the data the optimal solution to this dynamic programming problem. The algorithm is shown to consistently estimate the optimal policy that may choose different models based on a set of covariates. A typical example is the one of switching between different portfolio models under rebalancing costs, using macroeconomic information. Using a set of macroeconomic variables and price data, an empirical application to the aforementioned portfolio problem shows superior performance to choosing the best portfolio model with hindsight.

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper involves advanced stochastic dynamic programming, consistency proofs, and reinforcement learning algorithms, indicating high mathematical density. It also includes an empirical application with macroeconomic variables and price data, demonstrating backtest-ready implementation with performance comparisons.

Research Flowchart

  flowchart TD
  A["Research Goal: Dynamic Model Selection<br>Under Switching Costs"] --> B["Data Inputs: Macroeconomic Variables & Asset Prices"]
  B --> C["Methodology: RL Algorithm for<br>Dynamic Programming Approximation"]
  C --> D["Computational Process:<br>Estimate Optimal State-Dependent Policy"]
  D --> E["Outcome: Superior Performance vs<br>Hindsight Optimal Model Selection"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a reinforcement learning algorithm to dynamically select among portfolio models accounting for rebalancing costs.

Source sentence
A Reinforcement learning algorithm is used to approximate and estimate from the data the optimal solution to this dynamic programming problem.

Assumes 5

  • State‑action pairs are constructed with a bounded action set and exogenous state component. data
    Source sentence
    Assumption 1 (State Action Pair) The state action pair (St, At)t≥1 and A0 satisfy (5) where |A| is a bounded integer, St = (˜St, At−1) where ˜St takes values in some set ˜S, and (˜St)t≥1 is generated independently of (At)t≥0.
  • The state process is a strictly stationary Markov chain with exponentially decaying beta‑mixing coefficients. statistical
    Source sentence
    Assumption 2 (Markov Assumption) The sequence of states S := (St)≥1 is a strictly stationary Markov process with beta mixing coefficients (βq)q≥1 satisfying βq ≲e−Cβq for some constant Cβ > 0.
  • Rewards have finite moments of order at least two. statistical
    Source sentence
    Assumption 3 (Moments) We have that maxa∈A |R (T a·; θa (·))|v,P < ∞ for v ≥2.
  • Reward‑parameter maps θa(s) are independent of the state sequence and measurable at time zero. model
    Source sentence
    →θa (s) is possibly random, but independent of the sequence of states (St)t≥1 and measurable at time t = 0.
Show 1 more
  • The approximating function class is a K‑dimensional linear span of basis functions. model
    Source sentence
    Assumption 5 (Approximating Function Class) The function space in (6) with functions {φl : l = 1, 2, ..., K} is a K dimensional vector space or a subset of it.

Data

  • S&P 500 stock constituents · Kaggle market-prices public
    S&P 500 constituents
    Source sentence
    The data for the stock constituents of the S&P500 were downloaded from Kaggle https://www.kaggle.com/datasets/andrewmvd/sp-500-stocks?resource=download&select=sp500_companies.csv.

Recognised sources: Kaggle

Costs

Modeled Costs are part of the reported headline results.

Costs are set equal to a constant times the relative turnover, where relative turnover is the ℓ1 norm of the change in portfolio weights.

Source sentence
Transaction costs can be set equal to a constant times the relative turnover. The relative turnover is the ℓ1 norm of wa −wa′.

Finds 3

  • The RL policy yields higher average daily rewards and annualized net returns than both the Greedy and Fixed strategies; e.g., Greedy’s max annualized net return is about 12% versus 22% for RL and 17% for Fixed 0. performance
    Source sentence
    Greedy has a maximum annualized net return of only about the 12% compared to 22% and 17% for RL and Fixed 0.
  • RL consistently provides higher average reward than any of the fixed strategies across cost levels. performance
    Source sentence
    On the other hand, RL consistently provides higher average reward than any of the fixed strategies.
  • In simulations, RL outperforms Greedy especially under large transaction costs, where fixed models can have negative rewards but RL remains positive. performance
    Source sentence
    Overall, RL performs better than Greedy. For large transaction costs, some of the fixed models provide average negative rewards, while RL always has an average positive reward.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 20 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.