Paper: arXiv 2502.01992
Authors: Arnav Grover
Abstract
In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF). Our framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals. Traditional LLMs, while competent in sentiment analysis, lack contextual alignment for financial market applications. To bridge this gap, we fine-tune the LLaMA-3.2-3B-Instruct model using a custom RLMF prompt design that integrates historical market data and reward-based feedback. Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II. You can find the code for this project on GitHub.
Complexity vs Empirical Score
- Math Complexity: 4.0/10
- Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs not modelled; see penalty flags)
- Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 7.0)
Why this score: The paper uses standard deep learning and reinforcement learning concepts without advanced mathematical derivations, yet it presents a complete experimental pipeline with dataset splits, performance metrics, and a GitHub code link.
Research Flowchart
flowchart TD A["Research Goal: Improve LLM-Engineered<br>Trading Signals Challenge"] --> B["Data Input: Historical<br>Market Data & Features"] B --> C["Methodology: RLMF Prompt Framework<br>RL from Market Feedback"] C --> D["Computational Process:<br>Fine-tune LLaMA-3.2-3B-Instruct"] D --> E["Output: Precise<br>Trading Signals"] E --> F["Findings: Winner of Task II<br>Superior Consistency & Outcomes"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a novel prompt framework for fine‑tuning large language models with Reinforcement Learning from Market Feedback (RLMF).
Source sentence
this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF).
Assumes 1
- The reward function gives positive rewards for long positions when a strong positive return is observed and penalizes when returns are negative despite positive sentiment, and similarly for short positions. model
Source sentence
the reward varies depending on the actual price movement: long positions are rewarded if a strong positive return is observed, while negative returns despite positive sentiment lead to penalties. Similarly, short positions are rewarded when negative
Data
2023 – 2023 · US equities · equities
- news headlines and stock price data with technical indicators text-news unknown
Source sentence
The experimental setup for testing and validating the proposed model’s performance spans 2020 to 2023, assessing the accuracy and profitability of sentiment-based trading signals against the baseline. This setup includes news headlines, stock price data, and technical indicators to effectively align sentiment scores wi
Costs
Not modeled
Finds 2
- The RLMF‑tuned framework outperforms baseline methods in signal consistency and achieves tighter trading outcomes, winning Task II of the FinRL Challenge. performance
Source sentence
Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II.
- Llama‑3.2‑3B‑Instruct shows higher variability in cumulative returns across tickers, while FinRLlama yields a more conservative approach with reduced volatility. performance
Source sentence
In summary, Llama-3.2-3B-Instruct displays a broader range of cumulative returns, indicating higher variability and greater individual gains and losses, while FinRLlama suggests a more conservative approach with reduced volatility in cumulative returns across tickers and smoother mean evaluation.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 16 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (113)
- When is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks (2011) — link
- A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks (2023) — link
- Reinforcement Learning for Quantitative Trading (2023) — link
- Predicting Economic Trends and Stock Market Prices with Deep Learning and Advanced Machine Learning Techniques (2023) — link
- Sentiment Analysis on Financial News Using Pre-trained Neural Language Models and Domain Sentiment Dictionary (2020) — link
- Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022) — link
- Prompt Engineering Strategies for Effective Large Language Model Use (2023) — link
- Beyond Fine-Tuning: Optimal Prompt Design for Domain-Specific Applications (2023) — link
and 105 more
- Twitter mood predicts the stock market (2011) — link
- Incorporating Historical Price Data for Financial Predictions Using Neural Networks (2017) — link
- Dynamic Prediction of Stock Movements Using Machine Learning and Historical Data (2011) — link
- Technical Analysis: The Complete Resource for Financial Market Technicians (2010)
- BERT: Pre-training of deep bidirectional transformers for language understanding (2019) — link
- Financial text mining and its applications to market prediction (2021) — link
- Deep reinforcement learning for financial trading (2016) — link
- Human-level control through deep reinforcement learning (2015) — link
- Mastering the game of Go with deep neural networks and tree search (2016) — link
- FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning (2022) — link
- FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (2020) — link
- Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024 (2024) — link
- A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges — → 2406.11903
- The patent holder’s dilemma: Buy, sell, or troll? (2007) — link
- Deciding equivalances among conjunctive aggregate queries (2007) — link
- Special issue: Digital Libraries (1996)
- Understanding Policy-Based Networking (2001)
- First-Order Dynamic Logic (1979) — link
- The title of book one (2007) — link
- The title of book two (2008) — link
- Achieving application requirements (1990) — link
- Statecarts in use: structured analysis and object-orientation (1998) — link
- The Art of Computer Programming, Vol. 1: Fundamental Algorithms (3rd. ed.) (1997)
- The Art of Computer Programming (1998)
- An experiment in bibliographic mark-up: Parsing metadata for XML export (2010) — link
- Catch me, if you can: Evading network signatures with web-based polymorphic worms (2007)
- Catch me, if you can: Evading network signatures with web-based polymorphic worms (2008)
- Catch me, if you can: Evading network signatures with web-based polymorphic worms (2009)
- Predicate Path expressions (1979) — link
- LOGICS of Programs: AXIOMATICS and DESCRIPTIVE POWER (1978)
- Optimal Motion Control of a Ground Vehicle (2003)
- Algorithms for Closest-Point Problems (Computational Geometry) (1985)
- Introduction to Bayesian Statistics (2001) — link
- CLIFFORD: a Maple 11 Package for Clifford Algebra Computations, version 11 (2007) — link
- Stats and Analysis (2006) — link
- A more perfect union (2008) — link
- The fountain of youth (2009)
- Solder man (2003) — link
- Interview with Bill Kinder: January 13, 2005 (2005) — link
- The Enabling of Digital Libraries (2008)
- (old) Finding minimum congestion spanning trees (2000) — link
- (new) Finding minimum congestion spanning trees (2000) — link
- (old) Distributed data source verification in wireless sensor networks (2009) — link
- (new) Distributed data source verification in wireless sensor networks (2009) — link
- Portalis: using competitive online interactions to support aid initiatives for the homeless (2008) — link
- Visual Basic 6: Design, Specification, and Objects with Other (1999)
- The Latex Web Companion: Integrating TEX, HTML, and XML (1999)
- Vertex Types in Book-Embeddings (1987)
- Vertex Types in Book-Embeddings (1987)
- CHI ‘08: CHI ‘08 extended abstracts on Human factors in computing systems (2008)
- Algorithms for Closest-Point Problems (Computational Geometry) (1985)
- IEEE TCSC Executive Committee (2004) — link
- Distributed systems (2nd Ed.) (1993)
- New Algorithms for Dependency-Directed Backtracking (Master’s thesis) (1986)
- New Algorithms for Dependency-Directed Backtracking (Master’s thesis) (1986)
- Seminumerical Algorithms (1981)
- E-commerce and cultural values (2001) — link
- Chapter 9 (2002) — link
- The implementation of electronic commerce in SMEs in Singapore (Incoll) (2003) — link
- E-commerce and cultural values - (InBook-num-in-chap) (2004) — link
- E-commerce and cultural values (Inbook-text-in-chap) (2005) — link
- E-commerce and cultural values (Inbook-num chap) (2006) — link
- A library-based synthesis methodology for reversible logic (2010)
- Synthesis of Reversible Circuit Using Cycle-Based Approach (2010)
- Algorithmic Enumeration of Ideal Classes for Quaternion Orders (2010) — link
- Chapter II: Notes on data structuring (1972) — link
- Transcript of question and answer session (1981) — link
- Go to statement considered harmful (1979) — link
- Three-dimensional virtual acoustic displays (1992) — link
- Managerial expert systems and organizational change: some critical research issues (1987) — link
- Simplified Structured COBOL with Microsoft/MicroFocus COBOL (1990)
- A multi-radio unification protocol for IEEE 802.11 wireless networks (2004)
- Wireless Sensor Networks: A Survey (2002)
- A Survey on Wireless Multimedia Sensor Networks (2007)
- SSCH : Slotted Seeded Channel Hopping for Capacity Improvement in IEEE 802.11 Ad-Hoc Wireless Networks (2004)
- XBOW Sensor Motes Specifications (2008)
- Overview of Sensor Networks (2004)
- CodeBlue : Sensor Networks for Medical Care (2008)
- Investigating Network Architectures for Body Sensor Networks (2007)
- Channel-Hopping Multiple Access (2000)
- Body Sensor Networks (2008)
- Mapping Powerlists onto Hypercubes (1994)
- Automatic Parallelization for Distributed-Memory Multiprocessing Systems (1989)
- User recovery and reversal in interactive systems (1984)
- Generalizing specifications for uniformly implemented loops (1985)
- Towards monolingual programming environments (1985)
- The \TeX book (1984)
- Distributed algorithms for finding centers and medians in networks (1984)
- \it \LaTeX : A Document Preparation System (1986)
- Program transformations in a denotational setting (1985)
- Seminumerical Algorithms (1981)
- A high-level approach to computer document formatting (1980)
- A multifrequency MAC specially designed for wireless sensor network applications (2010) — link
- Institutional members of the \TeX Users Group (2017) — link
- acmart— C lass for typesetting publications of ACM (2017) — link
- Reasoning About Naming Systems (1993) — link
- Babel, a Multilingual Style-Option System for Use with LaTeX’s Standard Document Styles (1991)
- A Methodology for Implementing Highly Concurrent Data Objects (1993) — link
- Publication quality tables in \LaTeX (2005)
- Using the amsthm Package (2015)
- R: A Language and Environment for Statistical Computing (2019) — link
- UMass Citation Field Extraction Dataset (2013) — link
- Citation concept analysis (CCA) —A new form of citation analysis revealing the usefulness of concepts for other researchers illustrated by two exemplary case studies including classic books by Thomas S.~Kuhn and Karl R.~Popper (2019) — link
- Learning Soft Linear Constraints with Application to Citation Field Extraction (2014) — link
- Maintaining Discrete Probability Distributions Optimally (1993)
1 of 113 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.