{"paper_id":"2501.17992","title":"Reinforcement-Learning Portfolio Allocation with Dynamic Embedding of Market Information","authors":["Jinghai He","Cheng Hua","Chunyang Zhou","Zeyu Zheng"],"paper_date":"2025-01-29","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2501.17992v1","page_url":"https://thequant.space/flowcharts/reinforcement-learning-portfolio-allocation-with-dynamic-emb/","scores":{"empirical_rigor":7,"math_complexity":8,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes an end‑to‑end Dynamic Embedding Reinforcement Learning (DERL) framework that combines deep RL, generative autoencoders and online meta‑learning for portfolio allocation.","quote":"In this paper, to address the challenges of high-dimensional portfolio allocation in a dynamic non-stationary market, we propose an end-to-end framework named Dynamic Embedding Reinforcement Learning (DERL), which leverages three deep learning methods—deep reinforcement learning, generative encoders, and meta‑learning.","summary":"We develop a portfolio allocation framework that leverages deep learning techniques to address challenges arising from high-dimensional, non-stationary, and low-signal-to-noise market information. Our approach includes a dynamic embedding method that reduces the non-stationary, high-dimensional…","strategy":"DERL (Dynamic Embedding RL) portfolio"},"assumptions":{"status":"available","items":[{"assumption":"No leverage or short selling is allowed.","type":"execution","stated":"explicit","quote":"Finally, we assume no leverage or short selling is allowed, aligning our strategy with the constraints typically encountered in mutual fund portfolio management."},{"assumption":"Investor maximizes Sharpe ratio of net returns after transaction costs.","type":"model","stated":"explicit","quote":"The investor is assumed to maximize the Sharpe ratio of net portfolio returns after accounting for transaction costs."},{"assumption":"Transaction cost rate of 0.1% applied to each trade.","type":"execution","stated":"explicit","quote":"cost rate of 0.1% is applied to the total value of each trade."},{"assumption":"Portfolio is rebalanced daily using only price‑volume data and technical indicators.","type":"data","stated":"explicit","quote":"our method relies on daily adjustments solely based on price‑volume data and technical indicators."},{"assumption":"Only the top 500 U.S. stocks by market capitalization are considered.","type":"data","stated":"explicit","quote":"we evaluate out‑of‑sample portfolio performance using top 500 stocks in terms of market capitalization in each subperiod."},{"assumption":"Sharpe ratio is used as the RL reward function.","type":"model","stated":"explicit","quote":"we use the Sharpe ratio to measure portfolio performance, which serves as the final reward for the reinforcement learning agent."}]},"datasets":{"status":"available","items":[{"name":"CRSP daily stock file","provider":"CRSP (Center for Research in Security Prices)","kind":"market-prices","access":"commercial","period":"January 1, 1990 to December 31, 2022","universe":"top 500 U.S. stocks by market value","quote":"We evaluate our model performance using the top 500 stocks by market value, which are actively traded. The trading information for each constituent stock, including daily open (O), high (H), low (L), close (C) prices, trading volumes (V), and returns, is collected from the CRSP (Center for Research in Security Prices) ","known_id":"crsp"}],"known":[{"id":"cboe","name":"Cboe","provider":"Cboe Global Markets","kind":"options","access":"commercial","quote":"We use the CBOE VIX, calculated based on the prices of S&P 500 index options, to measure the market volatility.","mentions":3,"used_in":1},{"id":"crsp","name":"CRSP","provider":"Center for Research in Security Prices","kind":"market-prices","access":"commercial","quote":"The trading information for each constituent stock, including daily open (O), high (H), low (L), close (C) prices, trading volumes (V), and returns, is collected from the CRSP (Center for Research in Security Prices) database, covering the period from Jan-uary 1, 1990, to December 31, 2022.","mentions":2,"used_in":1}],"sample":{"start":"1993","end":"2022","market":"top 500 US stocks","asset_class":"equities","frequency":"daily"},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":true,"detail":"0.1% of the total trade value is charged as transaction cost.","quote":"cost rate of 0.1% is applied to the total value of each trade.","source":"front_matter","rule":"R3","candidates":{"front_matter":"yes","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"0.1% of the total trade value is charged as transaction cost.","card_quote":"cost rate of 0.1% is applied to the total value of each trade.","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"DERL achieves significantly higher Sharpe and Sortino ratios than the two‑step predict‑then‑optimize method and both value‑ and equal‑weighted portfolios.","kind":"performance","quote":"Empirical results show that our DERL framework achieves significantly higher Sharpe and Sortino ratios compared to the two-step predict‑then‑optimize (PTO) method using machine learning models, as well as value- and equal-weighted portfolios."},{"finding":"DERL’s outperformance is especially strong during high‑volatility regimes.","kind":"performance","quote":"The results demonstrate that DERL’s outperformance is highly significant under high market volatility conditions compared to low‑volatility conditions."},{"finding":"The daily risk‑adjusted return alpha of DERL exceeds 0.03%, i.e., about 7.5% per annum, and cannot be fully explained by standard factor models.","kind":"performance","quote":"Factor analysis shows that the performance of the DERL framework cannot be fully explained by the Fama and French (1993) three‑factor model or Fama and French (1993)‑Carhart (1997) four‑factor model, with the daily risk‑adjusted return α exceeding 0.03%, or 7.5% per an…"},{"finding":"Ablation studies confirm that the framework’s performance is robust across different RL algorithms and that embedding and meta‑learning are crucial components.","kind":"performance","quote":"All three RL algorithms within the DERL framework outperform versions without dynamic updati…"},{"finding":"The DERL agent exhibits volatility‑timing capability, reducing market exposure during periods of market stress.","kind":"performance","quote":"Further analysis reveals that the DERL agent has volatility timing capability, reducing"}],"reported_performance":{"strategy":"DERL (Dynamic Embedding RL) portfolio","sharpe":1.0407,"sharpe_benchmark":0.4802,"annual_return_pct":14.81,"total_return_pct":null,"max_drawdown_pct":null},"out_of_sample":{"method":"walk-forward","headline_out_of_sample":true}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":66,"citations":{"cited_by_in_corpus":1,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":1},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:46:25Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}