Paper: arXiv 2502.01992

Authors: Arnav Grover

Abstract

In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF). Our framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals. Traditional LLMs, while competent in sentiment analysis, lack contextual alignment for financial market applications. To bridge this gap, we fine-tune the LLaMA-3.2-3B-Instruct model using a custom RLMF prompt design that integrates historical market data and reward-based feedback. Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II. You can find the code for this project on GitHub.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 7.0)

Why this score: The paper uses standard deep learning and reinforcement learning concepts without advanced mathematical derivations, yet it presents a complete experimental pipeline with dataset splits, performance metrics, and a GitHub code link.

Research Flowchart

  flowchart TD
  A["Research Goal: Improve LLM-Engineered<br>Trading Signals Challenge"] --> B["Data Input: Historical<br>Market Data & Features"]
  B --> C["Methodology: RLMF Prompt Framework<br>RL from Market Feedback"]
  C --> D["Computational Process:<br>Fine-tune LLaMA-3.2-3B-Instruct"]
  D --> E["Output: Precise<br>Trading Signals"]
  E --> F["Findings: Winner of Task II<br>Superior Consistency & Outcomes"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a novel prompt framework for fine‑tuning large language models with Reinforcement Learning from Market Feedback (RLMF).

Source sentence
this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF).

Assumes 1

  • The reward function gives positive rewards for long positions when a strong positive return is observed and penalizes when returns are negative despite positive sentiment, and similarly for short positions. model
    Source sentence
    the reward varies depending on the actual price movement: long positions are rewarded if a strong positive return is observed, while negative returns despite positive sentiment lead to penalties. Similarly, short positions are rewarded when negative

Data

Backtest sample: 2023 – 2023 · US equities · equities

  • news headlines and stock price data with technical indicators text-news unknown
    2020-2023
    Source sentence
    The experimental setup for testing and validating the proposed model’s performance spans 2020 to 2023, assessing the accuracy and profitability of sentiment-based trading signals against the baseline. This setup includes news headlines, stock price data, and technical indicators to effectively align sentiment scores wi

Costs

Not modeled A trading or execution result is reported without costs.

Finds 2

  • The RLMF‑tuned framework outperforms baseline methods in signal consistency and achieves tighter trading outcomes, winning Task II of the FinRL Challenge. performance
    Source sentence
    Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II.
  • Llama‑3.2‑3B‑Instruct shows higher variability in cumulative returns across tickers, while FinRLlama yields a more conservative approach with reduced volatility. performance
    Source sentence
    In summary, Llama-3.2-3B-Instruct displays a broader range of cumulative returns, indicating higher variability and greater individual gains and losses, while FinRLlama suggests a more conservative approach with reduced volatility in cumulative returns across tickers and smoother mean evaluation.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 16 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (113)

  1. When is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks (2011) — link
  2. A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks (2023) — link
  3. Reinforcement Learning for Quantitative Trading (2023) — link
  4. Predicting Economic Trends and Stock Market Prices with Deep Learning and Advanced Machine Learning Techniques (2023) — link
  5. Sentiment Analysis on Financial News Using Pre-trained Neural Language Models and Domain Sentiment Dictionary (2020) — link
  6. Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022) — link
  7. Prompt Engineering Strategies for Effective Large Language Model Use (2023) — link
  8. Beyond Fine-Tuning: Optimal Prompt Design for Domain-Specific Applications (2023) — link
and 105 more
  1. Twitter mood predicts the stock market (2011) — link
  2. Incorporating Historical Price Data for Financial Predictions Using Neural Networks (2017) — link
  3. Dynamic Prediction of Stock Movements Using Machine Learning and Historical Data (2011) — link
  4. Technical Analysis: The Complete Resource for Financial Market Technicians (2010)
  5. BERT: Pre-training of deep bidirectional transformers for language understanding (2019) — link
  6. Financial text mining and its applications to market prediction (2021) — link
  7. Deep reinforcement learning for financial trading (2016) — link
  8. Human-level control through deep reinforcement learning (2015) — link
  9. Mastering the game of Go with deep neural networks and tree search (2016) — link
  10. FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning (2022) — link
  11. FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (2020) — link
  12. Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024 (2024) — link
  13. A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges — → 2406.11903
  14. The patent holder’s dilemma: Buy, sell, or troll? (2007) — link
  15. Deciding equivalances among conjunctive aggregate queries (2007) — link
  16. Special issue: Digital Libraries (1996)
  17. Understanding Policy-Based Networking (2001)
  18. First-Order Dynamic Logic (1979) — link
  19. The title of book one (2007) — link
  20. The title of book two (2008) — link
  21. Achieving application requirements (1990) — link
  22. Statecarts in use: structured analysis and object-orientation (1998) — link
  23. The Art of Computer Programming, Vol. 1: Fundamental Algorithms (3rd. ed.) (1997)
  24. The Art of Computer Programming (1998)
  25. An experiment in bibliographic mark-up: Parsing metadata for XML export (2010) — link
  26. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2007)
  27. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2008)
  28. Catch me, if you can: Evading network signatures with web-based polymorphic worms (2009)
  29. Predicate Path expressions (1979) — link
  30. LOGICS of Programs: AXIOMATICS and DESCRIPTIVE POWER (1978)
  31. Optimal Motion Control of a Ground Vehicle (2003)
  32. Algorithms for Closest-Point Problems (Computational Geometry) (1985)
  33. Introduction to Bayesian Statistics (2001) — link
  34. CLIFFORD: a Maple 11 Package for Clifford Algebra Computations, version 11 (2007) — link
  35. Stats and Analysis (2006) — link
  36. A more perfect union (2008) — link
  37. The fountain of youth (2009)
  38. Solder man (2003) — link
  39. Interview with Bill Kinder: January 13, 2005 (2005) — link
  40. The Enabling of Digital Libraries (2008)
  41. (old) Finding minimum congestion spanning trees (2000) — link
  42. (new) Finding minimum congestion spanning trees (2000) — link
  43. (old) Distributed data source verification in wireless sensor networks (2009) — link
  44. (new) Distributed data source verification in wireless sensor networks (2009) — link
  45. Portalis: using competitive online interactions to support aid initiatives for the homeless (2008) — link
  46. Visual Basic 6: Design, Specification, and Objects with Other (1999)
  47. The Latex Web Companion: Integrating TEX, HTML, and XML (1999)
  48. Vertex Types in Book-Embeddings (1987)
  49. Vertex Types in Book-Embeddings (1987)
  50. CHI ‘08: CHI ‘08 extended abstracts on Human factors in computing systems (2008)
  51. Algorithms for Closest-Point Problems (Computational Geometry) (1985)
  52. IEEE TCSC Executive Committee (2004) — link
  53. Distributed systems (2nd Ed.) (1993)
  54. New Algorithms for Dependency-Directed Backtracking (Master’s thesis) (1986)
  55. New Algorithms for Dependency-Directed Backtracking (Master’s thesis) (1986)
  56. Seminumerical Algorithms (1981)
  57. E-commerce and cultural values (2001) — link
  58. Chapter 9 (2002) — link
  59. The implementation of electronic commerce in SMEs in Singapore (Incoll) (2003) — link
  60. E-commerce and cultural values - (InBook-num-in-chap) (2004) — link
  61. E-commerce and cultural values (Inbook-text-in-chap) (2005) — link
  62. E-commerce and cultural values (Inbook-num chap) (2006) — link
  63. A library-based synthesis methodology for reversible logic (2010)
  64. Synthesis of Reversible Circuit Using Cycle-Based Approach (2010)
  65. Algorithmic Enumeration of Ideal Classes for Quaternion Orders (2010) — link
  66. Chapter II: Notes on data structuring (1972) — link
  67. Transcript of question and answer session (1981) — link
  68. Go to statement considered harmful (1979) — link
  69. Three-dimensional virtual acoustic displays (1992) — link
  70. Managerial expert systems and organizational change: some critical research issues (1987) — link
  71. Simplified Structured COBOL with Microsoft/MicroFocus COBOL (1990)
  72. A multi-radio unification protocol for IEEE 802.11 wireless networks (2004)
  73. Wireless Sensor Networks: A Survey (2002)
  74. A Survey on Wireless Multimedia Sensor Networks (2007)
  75. SSCH : Slotted Seeded Channel Hopping for Capacity Improvement in IEEE 802.11 Ad-Hoc Wireless Networks (2004)
  76. XBOW Sensor Motes Specifications (2008)
  77. Overview of Sensor Networks (2004)
  78. CodeBlue : Sensor Networks for Medical Care (2008)
  79. Investigating Network Architectures for Body Sensor Networks (2007)
  80. Channel-Hopping Multiple Access (2000)
  81. Body Sensor Networks (2008)
  82. Mapping Powerlists onto Hypercubes (1994)
  83. Automatic Parallelization for Distributed-Memory Multiprocessing Systems (1989)
  84. User recovery and reversal in interactive systems (1984)
  85. Generalizing specifications for uniformly implemented loops (1985)
  86. Towards monolingual programming environments (1985)
  87. The \TeX book (1984)
  88. Distributed algorithms for finding centers and medians in networks (1984)
  89. \it \LaTeX : A Document Preparation System (1986)
  90. Program transformations in a denotational setting (1985)
  91. Seminumerical Algorithms (1981)
  92. A high-level approach to computer document formatting (1980)
  93. A multifrequency MAC specially designed for wireless sensor network applications (2010) — link
  94. Institutional members of the \TeX Users Group (2017) — link
  95. acmart— C lass for typesetting publications of ACM (2017) — link
  96. Reasoning About Naming Systems (1993) — link
  97. Babel, a Multilingual Style-Option System for Use with LaTeX’s Standard Document Styles (1991)
  98. A Methodology for Implementing Highly Concurrent Data Objects (1993) — link
  99. Publication quality tables in \LaTeX (2005)
  100. Using the amsthm Package (2015)
  101. R: A Language and Environment for Statistical Computing (2019) — link
  102. UMass Citation Field Extraction Dataset (2013) — link
  103. Citation concept analysis (CCA) —A new form of citation analysis revealing the usefulness of concepts for other researchers illustrated by two exemplary case studies including classic books by Thomas S.~Kuhn and Karl R.~Popper (2019) — link
  104. Learning Soft Linear Constraints with Application to Citation Field Extraction (2014) — link
  105. Maintaining Discrete Probability Distributions Optimally (1993)

1 of 113 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.