Paper: arXiv 2410.18988
Authors: Joel R. Bock
Abstract
This paper describes experiments on fine-tuning a small language model to generate forecasts of long-horizon stock price movements. Inputs to the model are narrative text from 10-K reports of large market capitalization companies in the S&P 500 index; the output is a forward-looking buy or sell decision. Price direction is predicted at discrete horizons up to 12 months after the report filing date. The results reported here demonstrate good out-of-sample statistical performance (F1-macro= 0.62) at medium to long investment horizons. In particular, the buy signals generated from 10-K text are found most precise at 6 and 9 months in the future. As measured by the F1 score, the buy signal provides between 4.8 and 9 percent improvement against a random stock selection model. In contrast, sell signals generated by the models do not perform well. This may be attributed to the highly imbalanced out-of-sample data, or perhaps due to management drafting annual reports with a bias toward positive language. Cross-sectional analysis of performance by economic sector suggests that idiosyncratic reporting styles within industries are correlated with varying degrees and time scales of price movement predictability.
Complexity vs Empirical Score
- Math Complexity: 3.5/10
- Empirical Rigor: 7.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper employs standard NLP/ML techniques (fine-tuning a transformer model) with relatively simple math, but demonstrates strong empirical rigor through detailed backtesting methodology, out-of-sample cross-validation, and reported performance metrics (F1 scores) on real financial data.
Research Flowchart
flowchart TD A["Research Goal: Forecast long-horizon stock price moves<br>using NLP on 10-K reports"] --> B["Data Input:<br>10-K Reports from S&P 500"] B --> C["Preprocessing & Tokenization"] C --> D["Model: Small Language Model<br>Fine-tuned for Buy/Sell Prediction"] D --> E["Forecast Horizons: 3, 6, 9, 12 Months"] E --> F["Outcomes"] F --> G["Buy Signals: High Precision<br>F1-macro = 0.62<br>Best at 6 & 9 months"] F --> H["Sell Signals: Low Performance<br>Attributed to Data Imbalance<br>and Management Bias"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper fine‑tunes a small language model on 10‑K report text to generate forward‑looking buy or sell decisions for S&P 500 stocks.
Source sentence
In the present study, a pre-trained language model is fine‑tuned [18] to generate forecasts of future stock price movements. Inputs to the model are raw text from 10‑K reports of large market capitalization companies in the S&P 500 index; the output is a forward‑looking buy or sell decision.
Assumes 4
- Only narrative text from 10‑K filings is used as input; no structured financial data are included. data
Source sentence
No structured financial input data are used; numeric data enters only as it appears in the narrative sections of the report.
- Companies are split by SEC ID so that the same firm does not appear in both training and test sets. execution
Source sentence
The data were partitioned by ID number such that companies were not simultaneously represented in both training and out‑of‑sample test sets during the experiments.
- Out‑of‑sample performance is evaluated without accounting for trading costs or market impact. execution
Source sentence
The results reported here demonstrate good out‑of‑sample statistical performance (F1‑macro= 0.62) at medium to long investment horizons.
- The random‑selection benchmark assumes a null hypothesis of no predictive value in the 10‑K text. statistical
Source sentence
The null hypothesis of “no predictive value” in the 10‑K text was tested by running 2500 trials using a pseudo‑random number as a decision function.
Data
- 10‑K report text text-news public
Source sentence
Inputs to the model are narrative text from 10‑K reports of large market capitalization companies in the S&P 500 index.
Costs
Not applicable
Finds 5
- The model achieves an out‑of‑sample macro‑averaged F1 score of 0.62 for buy/sell classification across horizons up to 12 months. predictive-accuracy
Source sentence
The results reported here demonstrate good out‑of‑sample statistical performance (F1‑macro= 0.62) at medium to long investment horizons.
- Buy signals are most precise at 6‑ and 9‑month horizons, with the peak sensitivity at 9 months. performance
Source sentence
In particular, the “buy” signals generated from 10‑K text are found most precise at 6 and 9 months in the future
- The buy signal improves over random selection by 4.8 % to 9 % in aggregate, with the largest gain at the 9‑month horizon. performance
Source sentence
As measured by the F1 score, the buy signal provides between 4.8 and 9 percent improvement against a random stock selection model.
- Sell signals perform poorly, likely due to highly imbalanced out‑of‑sample data and positive‑language bias in reports. negative
Source sentence
In contrast, sell signals generated by the models do not perform well. This may be attributed to the highly imbalanced out‑of‑sample data, or perhaps due to management drafting annual reports with a bias toward positive language.
- Cross‑sectional analysis shows that reporting style effects on predictability vary by economic sector. statistical
Source sentence
Cross‑sectional analysis of performance by economic sector suggests that characteristic reporting styles within industries are correlated with varying degrees and time scales of price movement predictability.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 23 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.