Paper: arXiv 2507.01970
Authors: Ayaan Qayyum
Abstract
This paper will discuss how headline data can be used to predict stock prices. The stock price in question is the SPDR S&P 500 ETF Trust, also known as SPY that tracks the performance of the largest 500 publicly traded corporations in the United States. A key focus is to use news headlines from the Wall Street Journal (WSJ) to predict the movement of stock prices on a daily timescale with OpenAI-based text embedding models used to create vector encodings of each headline with principal component analysis (PCA) to exact the key features. The challenge of this work is to capture the time-dependent and time-independent, nuanced impacts of news on stock prices while handling potential lag effects and market noise. Financial and economic data were collected to improve model performance; such sources include the U.S. Dollar Index (DXY) and Treasury Interest Yields. Over 390 machine-learning inference models were trained. The preliminary results show that headline data embeddings greatly benefit stock price prediction by at least 40% compared to training and optimizing a machine learning system without headline data embeddings.
Complexity vs Empirical Score
- Math Complexity: 3.0/10
- Empirical Rigor: 7.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper uses practical ML techniques like PCA and embeddings with substantial empirical testing (390 models, real WSJ/DXY data), but lacks advanced mathematical derivations, focusing on application over theoretical complexity.
Research Flowchart
flowchart TD A["Research Goal:<br>Forecast SPY Stock Prices<br>using News Headlines"] --> B["Data Collection:<br>WSJ Headlines &<br>Financial Indicators DXY, Treasury Yields"] B --> C["Feature Engineering:<br>OpenAI Text Embeddings<br>+ PCA for Dimensionality Reduction"] C --> D["Model Training:<br>390+ ML Inference Models<br>trained on historical data"] D --> E["Evaluation &<br>Comparison vs Baseline"] E --> F["Key Finding:<br>40% Improvement in<br>Prediction Accuracy with Headline Data"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes using OpenAI-based embeddings of Wall Street Journal headlines, reduced by PCA, to predict daily SPY price movements.
Source sentence
A key focus is to use news headlines from the Wall Street Journal (WSJ) to predict the movement of stock prices on a daily timescale with OpenAI-based text embedding models used to create vector encodings of each headline with principal component analysis (PCA) to exact the key features.
Assumes 3
- Days when the market is closed are assumed to have no impact on the model. market
Source sentence
Another assumption was that any gaps in the market where the market is closed are assumed not to have an impact.
- No prediction can be made on days without SPY data. data
Source sentence
The resulting decision was that no prediction can be made on days without data for the SPY, which translates to days the New York Stock Exchange (NYSE) is closed.
- PCA reduces high‑dimensional embedding sparsity and improves model generalization. model
Source sentence
The PCA step reduced the 1,535-dimension vector to a set of reduced dimensions to reduce issues related to data sparsity.
Data
- Wall Street Journal headline dataset text-news public
Source sentence
A publicly available data set containing headlines from the Wall Street Journal
- SPY daily price data market-prices public
Source sentence
The SPY’s daily opening and closing, volume, and volatility were used as financial indicators.
- Dollar Stability Index (DXY) macro unknown
Source sentence
These include the Dollar Stability Index (DXY) and the Federal Interest Yields Data.
- Federal Interest Yields macro unknown
Source sentence
These include the Dollar Stability Index (DXY) and the Federal Interest Yields Data.
Costs
Not applicable
Finds 3
- Including WSJ headline embeddings improves prediction performance by up to 40% over models without them. performance
Source sentence
The results showed that including relevant financial headlines from the Wall Street Journal significantly improved model prediction performance by up to 40%.
- Preliminary results indicate at least a 40% improvement in predictive performance versus models using only historical prices and macro indicators. predictive-accuracy
Source sentence
Preliminary results indicate that incorporating headline data embeddings significantly improves predictive performance by at least 40% compared to models relying solely on historical stock prices and economic indicators.
- Time‑independent modeling performed as well as or better than time‑dependent modeling. performance
Source sentence
The results of the time-independent operations proved to be just as good, if not better, as time-dependent ones.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 29 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.