{"paper_id":"2411.17900","title":"Pretrained LLM Adapted with LoRA as a Decision Transformer for Offline RL in Quantitative Trading","authors":["Suyeol Yun"],"paper_date":"2024-11-26","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2411.17900v1","page_url":"https://thequant.space/flowcharts/pretrained-llm-adapted-with-lora-as-a-decision-transformer-f/","scores":{"empirical_rigor":6,"math_complexity":6.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a Decision Transformer initialized with pre-trained GPT-2 weights and fine‑tuned using Low‑Rank Adaptation (LoRA) for offline reinforcement learning in quantitative trading.","quote":"We introduce a Decision Transformer initialized with pre-trained GPT-2 weights and fine-tuned using LoRA","summary":"Developing effective quantitative trading strategies using reinforcement learning (RL) is challenging due to the high risks associated with online interaction with live financial markets. Consequently, offline RL, which leverages historical market data without additional exploration, becomes…","strategy":"DT LoRA GPT-2"},"assumptions":{"status":"available","items":[{"assumption":"The offline RL setting assumes a fixed dataset of historical trajectories with no further market interaction during training.","type":"data","stated":"explicit","quote":"... without any additional exploration or interaction during training."},{"assumption":"The financial market can be modeled as a Markov Decision Process (MDP).","type":"market","stated":"explicit","quote":"In quantitative trading, the environment represents the financial market, and the agent’s actions correspond to trading decisions."}]},"datasets":{"status":"none_found","items":[],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":"equities","frequency":"daily"},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":false,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"no","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"DT‑LoRA‑GPT2 consistently ranks among the top performers across metrics, often achieving the highest cumulative return and Sharpe ratio among baselines.","kind":"performance","quote":"DT‑LoRA‑GPT2 consistently ranks among the top performers"},{"finding":"Under the DDPG expert agent, DT‑LoRA‑GPT2 attains a cumulative return of 47.98% versus 42.88% for the randomly initialized model.","kind":"performance","quote":"under DDPG, DT‑LoRA‑GPT2 attains a return of 47.98% versus 42.88% for the random initialization"},{"finding":"Under the TD3 expert agent, DT‑LoRA‑GPT2 achieves a Sharpe Ratio of 2.14, higher than the 2.06 of the randomly initialized counterpart.","kind":"performance","quote":"Under TD3, it achieves a Sharpe Ratio of 2.14 compared to 2.06 for the randomly initialized model"},{"finding":"DT‑LoRA‑GPT2 records a maximum drawdown of -8.42% under A2C, improving upon the random model’s -9.42%.","kind":"performance","quote":"Under A2C, DT‑LoRA‑GPT2 records an MDD of -8.42%, improving upon the random model’s -9.42%"},{"finding":"The model effectively handles sparse and delayed rewards and captures complex temporal dependencies inherent in financial data.","kind":"other","quote":"our model effectively handles the challenges of sparse and delayed rewards and captures the complex temporal dependencies inherent in financial data"}],"reported_performance":{"strategy":"DT LoRA GPT-2","sharpe":2.22,"sharpe_benchmark":null,"annual_return_pct":null,"total_return_pct":47.98,"max_drawdown_pct":9.47},"out_of_sample":{"method":"holdout","headline_out_of_sample":true}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":35,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/syyunn/finrl-dt","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:52:01Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}