{"paper_id":"2308.02820","title":"Reinforcement Learning for Financial Index Tracking","authors":["Xianhua Peng","Chenyin Gong","Xue Dong He"],"paper_date":"2023-08-05","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2308.02820v2","page_url":"https://thequant.space/flowcharts/reinforcement-learning-for-financial-index-tracking/","scores":{"empirical_rigor":7,"math_complexity":8,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a discrete-time infinite‑horizon dynamic formulation for index tracking and an extension of deep reinforcement learning to solve it.","quote":"We propose the first discrete-time infinite-horizon dynamic formulation of the financial index tracking problem under both return-based tracking error and value-based tracking error.","summary":"We propose the first discrete-time infinite-horizon dynamic formulation of the financial index tracking problem under both return-based tracking error and value-based tracking error. The formulation overcomes the limitations of existing models by incorporating the intertemporal dynamics of market…","strategy":"RL-based index tracking with cash injection/withdrawal"},"assumptions":{"status":"available","items":[{"assumption":"Portfolio weights are selected at time t and kept constant until the next rebalancing within the period [t, t+1).","type":"model","stated":"explicit","quote":"We assume that at each time t = 0, 1, 2, . . . , the fund manager selects portfolio weights wt ∈Ft, and then rebalances the portfolio at each rebalancing time t + k M , k = 0, 1, . . . , M −1, such that wt+ k M = wt for all k = 0, 1, . . . , M −1."},{"assumption":"The fund manager’s trading strategy is adapted to the filtration generated by market information Ft.","type":"market","stated":"explicit","quote":"Let the σ-field Ft represent the set of market information available at time t, i.e., Ft = σ{Is, p1,s, p2,s, . . . , pN,s, Xs; ∀s ≤t}, where Xs represents market information other than prices at time s."},{"assumption":"Transaction cost at each rebalancing is a known nonlinear function of trade volume with parameters from Interactive Brokers.","type":"execution","stated":"explicit","quote":"The transaction cost is ξ1i per share, with a minimum of ξ3i and a maximum of ξ2i of the trade value."},{"assumption":"Only one sample path of financial data is available; training uses random start dates to generate multiple episodes.","type":"data","stated":"explicit","quote":"The basic idea is to obtain enough quarterly or semiyearly data from daily data by randomly selecting the starting date of the first time period of each episode during training."}]},"datasets":{"status":"available","items":[{"name":"CRSP daily stock data","provider":"Center for Research in Security Prices, LLC","kind":"market-prices","access":"commercial","period":"1975-01-02 to 2021-12-31 (varies by inde","universe":"Constituent stocks of DJIA, S&P 500, S&P 500 EWI","quote":"The data of prices, trading volumes, and T-Bill rates are obtained from the Center for Research in Security Prices, LLC (CRSP).","known_id":"crsp"}],"known":[{"id":"crsp","name":"CRSP","provider":"Center for Research in Security Prices","kind":"market-prices","access":"commercial","quote":"The data of prices, trading volumes, and T-Bill rates are obtained from the Center for Research in Security Prices, LLC (CRSP).3 The VIX data are downloaded from the website 3If the closing price on a day was not available, CRSP records the average of bid and ask prices with a leading dash so as to distinguish it from an actual closing price.","mentions":2,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":"equities","frequency":"daily"},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":true,"detail":"Nonlinear cost per share ξ1i, proportional fee ξ2i (0.5%), and minimum fee ξ3i (1.0 USD) as used by Interactive Brokers; also includes SEC and FINRA regulatory fees.","quote":"The transaction cost in the US market also includes SEC transaction fee and FINRA trading activity fee.","source":"front_matter","rule":"R3","candidates":{"front_matter":"yes","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"Nonlinear cost per share ξ1i, proportional fee ξ2i (0.5%), and minimum fee ξ3i (1.0 USD) as used by Interactive Brokers; also includes SEC and FINRA regulatory fees.","card_quote":"The transaction cost in the US market also includes SEC transaction fee and FINRA trading activity fee.","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The RL method achieves smaller mean return tracking error than the benchmark on a 17‑year out‑of‑sample test.","kind":"performance","quote":"Our RL method achieves smaller mean return tracking error than a benchmark method on the testing set for return tracking of the indices."},{"finding":"For value tracking, the RL method reduces mean value tracking error by an order of magnitude and yields a positive mean cash withdraw.","kind":"performance","quote":"For value tracking of indices, our RL method not only achieves one magnitude smaller mean value tracking error than the benchmark method, but also produces a positive mean cash withdraw."},{"finding":"Out‑of‑sample evaluation uses a rolling‑window approach over a 17‑year testing period, longer than prior studies.","kind":"other","quote":"We are the first to test the out-of-sample performance of an index tracking strategy using a 17-year-long testing set, which is much longer than those in the existing literature."}],"reported_performance":{"strategy":"RL-based index tracking with cash injection/withdrawal","sharpe":null,"sharpe_benchmark":null,"annual_return_pct":null,"total_return_pct":null,"max_drawdown_pct":null},"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":15,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/dppalomar/sparseIndexTracking","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:23:43Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}