Paper: arXiv 2307.04754
Authors: Francesco Cordoni, Alessio Sancetta
Abstract
A model among many may only be best under certain states of the world. Switching from a model to another can also be costly. Finding a procedure to dynamically choose a model in these circumstances requires to solve a complex estimation procedure and a dynamic programming problem. A Reinforcement learning algorithm is used to approximate and estimate from the data the optimal solution to this dynamic programming problem. The algorithm is shown to consistently estimate the optimal policy that may choose different models based on a set of covariates. A typical example is the one of switching between different portfolio models under rebalancing costs, using macroeconomic information. Using a set of macroeconomic variables and price data, an empirical application to the aforementioned portfolio problem shows superior performance to choosing the best portfolio model with hindsight.
Complexity vs Empirical Score
- Math Complexity: 8.0/10
- Empirical Rigor: 7.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper involves advanced stochastic dynamic programming, consistency proofs, and reinforcement learning algorithms, indicating high mathematical density. It also includes an empirical application with macroeconomic variables and price data, demonstrating backtest-ready implementation with performance comparisons.
Research Flowchart
flowchart TD A["Research Goal: Dynamic Model Selection<br>Under Switching Costs"] --> B["Data Inputs: Macroeconomic Variables & Asset Prices"] B --> C["Methodology: RL Algorithm for<br>Dynamic Programming Approximation"] C --> D["Computational Process:<br>Estimate Optimal State-Dependent Policy"] D --> E["Outcome: Superior Performance vs<br>Hindsight Optimal Model Selection"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a reinforcement learning algorithm to dynamically select among portfolio models accounting for rebalancing costs.
Source sentence
A Reinforcement learning algorithm is used to approximate and estimate from the data the optimal solution to this dynamic programming problem.
Assumes 5
- State‑action pairs are constructed with a bounded action set and exogenous state component. data
Source sentence
Assumption 1 (State Action Pair) The state action pair (St, At)t≥1 and A0 satisfy (5) where |A| is a bounded integer, St = (˜St, At−1) where ˜St takes values in some set ˜S, and (˜St)t≥1 is generated independently of (At)t≥0.
- The state process is a strictly stationary Markov chain with exponentially decaying beta‑mixing coefficients. statistical
Source sentence
Assumption 2 (Markov Assumption) The sequence of states S := (St)≥1 is a strictly stationary Markov process with beta mixing coefficients (βq)q≥1 satisfying βq ≲e−Cβq for some constant Cβ > 0.
- Rewards have finite moments of order at least two. statistical
Source sentence
Assumption 3 (Moments) We have that maxa∈A |R (T a·; θa (·))|v,P < ∞ for v ≥2.
- Reward‑parameter maps θa(s) are independent of the state sequence and measurable at time zero. model
Source sentence
→θa (s) is possibly random, but independent of the sequence of states (St)t≥1 and measurable at time t = 0.
Show 1 more
- The approximating function class is a K‑dimensional linear span of basis functions. model
Source sentence
Assumption 5 (Approximating Function Class) The function space in (6) with functions {φl : l = 1, 2, ..., K} is a K dimensional vector space or a subset of it.
Data
- S&P 500 stock constituents market-prices public
Source sentence
The data for the stock constituents of the S&P500 were downloaded from Kaggle https://www.kaggle.com/datasets/andrewmvd/sp-500-stocks?resource=download&select=sp500_companies.csv.
Costs
Modeled
Costs are set equal to a constant times the relative turnover, where relative turnover is the ℓ1 norm of the change in portfolio weights.
Source sentence
Transaction costs can be set equal to a constant times the relative turnover. The relative turnover is the ℓ1 norm of wa −wa′.
Finds 3
- The RL policy yields higher average daily rewards and annualized net returns than both the Greedy and Fixed strategies; e.g., Greedy’s max annualized net return is about 12% versus 22% for RL and 17% for Fixed 0. performance
Source sentence
Greedy has a maximum annualized net return of only about the 12% compared to 22% and 17% for RL and Fixed 0.
- RL consistently provides higher average reward than any of the fixed strategies across cost levels. performance
Source sentence
On the other hand, RL consistently provides higher average reward than any of the fixed strategies.
- In simulations, RL outperforms Greedy especially under large transaction costs, where fixed models can have negative rewards but RL remains positive. performance
Source sentence
Overall, RL performs better than Greedy. For large transaction costs, some of the fixed models provide average negative rewards, while RL always has an average positive reward.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 20 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.