{"paper_id":"2311.04946","title":"Causal Inference on Investment Constraints and Non-stationarity in Dynamic Portfolio Optimization through Reinforcement Learning","authors":["Yasuhiro Nakayama","Tomochika Sawaki"],"paper_date":"2023-11-08","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2311.04946v1","page_url":"https://thequant.space/flowcharts/causal-inference-on-investment-constraints-and-non-stationar/","scores":{"empirical_rigor":6,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a dynamic asset allocation investment strategy using reinforcement learning that incorporates non‑stationarity and practical investment constraints.","quote":"we have developed a dynamic asset allocation investment strategy using reinforcement learning techniques.","summary":"In this study, we have developed a dynamic asset allocation investment strategy using reinforcement learning techniques. To begin with, we have addressed the crucial issue of incorporating non-stationarity of financial time series data into reinforcement learning algorithms, which is a significant…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Funding costs, transaction costs, and cash ratios are ignored.","type":"execution","stated":"explicit","quote":"We don’t take into account funding costs, transaction costs, and cash ratios."},{"assumption":"Rebalancing is performed daily.","type":"execution","stated":"explicit","quote":"the rebalancing frequency is daily, and transaction costs are not considered."},{"assumption":"Financial time‑series mean, variance, and covariance are not constant over time.","type":"market","stated":"explicit","quote":"mean, variance, and covariance are not constant over time and that several events have been historically observed"},{"assumption":"Correlation state variable is discretized into three bins using a ±0.2 threshold.","type":"model","stated":"explicit","quote":"The correlation coefficient state variable is divided into three states of positive correlation/no correlation/negative correlation with a threshold of ±0.2"},{"assumption":"Table‑based reinforcement learning (SARSA and Q‑learning) is sufficient for the problem.","type":"model","stated":"explicit","quote":"we utilize table-based reinforcement learning methods such as SARSA and Q‑learning."}]},"datasets":{"status":"available","items":[{"name":"S&P500 total return index (USD)","provider":"Bloomberg","kind":"market-prices","access":"commercial","period":"April 2000 to March 2023","universe":"single risky asset","quote":"We choose the total return index of the S & P500, denominated in USD, as the risky asset","known_id":"bloomberg"},{"name":"US Treasury Index total return (USD)","provider":"Bloomberg","kind":"market-prices","access":"commercial","period":"April 2000 to March 2023","universe":"single non‑risky asset","quote":"the total return index of US Treasury Index, also denominated in USD, as the non‑risky asset","known_id":"bloomberg"}],"known":[{"id":"bloomberg","name":"Bloomberg","provider":"Bloomberg L.P.","kind":"market-prices","access":"commercial","quote":"The data is sourced from Bloomberg, and the daily return is calculated for the analysis.","mentions":1,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":null,"detail":null,"quote":"We don’t take into account funding costs, transaction costs, and cash ratios.","source":"front_matter","rule":"R3","candidates":{"front_matter":"no","evidence_card":"no","evidence_card_quoted":true},"card_detail":null,"card_quote":"We don’t take into account funding costs, transaction costs, and cash ratios.","meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The non‑stationary model’s average Sharpe ratio from FY2001 to FY2022 is higher than that of the base model and the difference is statistically significant.","kind":"performance","quote":"The non‑stationary model's average Sharpe ratio from FY2001 to FY2022 is higher than that of the base model, and the difference is statistically significant."},{"finding":"The non‑stationary model also outperforms the median Sharpe ratio of the random model on average.","kind":"performance","quote":"The non‑stationary model also outperforms the median of the random model on average Sharpe ratios."},{"finding":"Unconstrained (no rebalancing frequency limit) yields the highest average Sharpe ratio; performance declines as the rebalancing interval is lengthened (weekly, bi‑weekly, monthly).","kind":"performance","quote":"As expected, the unconstrained condition exhibits the highest average Sharpe ratio, and performance gradually decreases as the interval is extended."},{"finding":"Higher signal accuracy increases the selection ratio of non‑risky assets, with a weaker but positive effect for risky assets.","kind":"performance","quote":"The rate of selection for non‑risky assets increases as signal accuracy improved, whereas for risky assets, the trend is discerned in the order of each Accuracy, albeit it is weaker than that for non‑risky assets."},{"finding":"In periods where signal accuracy varies, the proportion of assets selected according to the signal rises with higher accuracy for both risky and non‑risky assets.","kind":"performance","quote":"As the horizontal axis shifts rightward and the accuracy in one aspect becomes higher, the proportion of selecting the asset indicated by the signals increases. This outcome is confirmed for both risky and non‑risky assets."}],"reported_performance":null,"out_of_sample":{"method":"walk-forward","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[],"contradicts":[{"paper":{"paper_id":"2508.14999","title":"Investment Portfolio Optimization Based on Modern Portfolio Theory and Deep Learning Models","paper_date":"2025-08-20","authors":["Maciej Wysocki","Paweł Sakowski"],"empirical_rigor":8,"page_url":"https://thequant.space/flowcharts/investment-portfolio-optimization-based-on-modern-portfolio/"},"question":"What is the impact of rebalancing frequency on portfolio performance (e.g., Sharpe ratio)?","this_paper_finding":"Unconstrained (no rebalancing frequency limit) yields the highest average Sharpe ratio; performance declines as the rebalancing interval is lengthened (weekly, bi‑weekly, monthly).","other_paper_finding":"Strategies with less frequent rebalancing (e.g., 30‑, 60‑, 90‑day) outperform those with very short rebalancing periods.","other_paper_quote":"Strategies with less frequent rebalancing typically perform better than these with the shortest rebalancing windows across all considered methods","note":"A finds performance declines as rebalancing moves from no limit to weekly/bi‑weekly/monthly, while B finds 30‑, 60‑, 90‑day intervals outperform very short periods.","confidence":0.741}],"qualifies":[],"compared_with":24,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:19:57Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}