{"paper_id":"2311.14318","title":"On optimal tracking portfolio in incomplete markets: The reinforcement learning approach","authors":["Lijun Bo","Yijie Huang","Xiang Yu"],"paper_date":"2023-11-24","paper_type":"theoretical","paper_url":"https://arxiv.org/pdf/2311.14318v2","page_url":"https://thequant.space/flowcharts/on-optimal-tracking-portfolio-in-incomplete-markets-the-rei/","scores":{"empirical_rigor":4,"math_complexity":8.5,"quadrant":"Lab Rats"},"proposes":{"status":"available","statement":"The paper introduces an exploratory formulation with entropy regularization and develops a continuous-time q‑learning algorithm for optimal tracking portfolios in incomplete markets.","quote":"When model parameters are unknown, we introduce the exploratory formulation for the auxiliary control problem with entropy regularization and develop the continuous-time q-learning algorithm in models of reflected diffusion processes.","summary":"This paper studies an infinite horizon optimal tracking portfolio problem using capital injection in incomplete market models. The benchmark process is modelled by a geometric Brownian motion with zero drift driven by some unhedgeable risk. The relaxed tracking formulation is adopted where the fund…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Riskless interest rate is zero (r = 0).","type":"market","stated":"explicit","quote":"Assume that the riskless interest rate r = 0 that amounts to the change of"},{"assumption":"Benchmark follows a geometric Brownian motion with zero drift driven by unhedgeable risk.","type":"model","stated":"explicit","quote":"The benchmark process is modelled by a geometric Brownian motion with zero drift driven by some unhedgeable risk."},{"assumption":"State dynamics are reflected (Neumann boundary condition).","type":"model","stated":"explicit","quote":"When model parameters are known, we formulate the equivalent auxiliary control problem with reflected state dynamics, for which the classical solution of the HJB equation with Neumann boundary condition is obtained explicitly."},{"assumption":"Fund account with injected capital must always outperform the benchmark (relaxed tracking).","type":"execution","stated":"explicit","quote":"The relaxed tracking formulation is adopted where the fund account compensated by the injected capital needs to outperform the benchmark process at any time."},{"assumption":"Convergence of stochastic approximation as time step →0 and horizon →∞.","type":"statistical","stated":"explicit","quote":"we establish the convergence of the stochastic approximation algorithms for time discretization and horizon truncation when ∆t →0 and T →∞"},{"assumption":"Temperature parameter set to γ = ρ/d for explicit fixed‑point solution.","type":"model","stated":"explicit","quote":"the temperature parameter γ = ρ/d, the map I has an explicit fixed point π∗∈Π given by"}]},"datasets":{"status":"available","items":[{"name":"real market data","provider":null,"kind":"market-prices","access":"unknown","period":null,"universe":null,"quote":"we present the comparison between the q-learning algorithm and the maximum likelihood estimation (MLE) for some real market data to demonstrate the outperformance of our algorithm."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"no","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The q‑learning algorithm shows satisfactory performance in illustrative numerical examples.","kind":"performance","quote":"In some illustrative numerical example, we show the satisfactory performance of the q-learning algorithm."},{"finding":"Learned parameters converge very satisfactorily toward their true values.","kind":"theoretical","quote":"we illustrate the very satisfactory performance of the iteration convergence of learned parameters towards their true values."},{"finding":"The q‑learning algorithm outperforms maximum likelihood estimation on real market data.","kind":"performance","quote":"we present the comparison between the q-learning algorithm and the maximum likelihood estimation (MLE) for some real market data to demonstrate the outperformance of our algorithm."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":23,"citations":{"cited_by_in_corpus":2,"by_independent_authors":2,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":1},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:18:48Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}