Paper: arXiv 2412.07587
Authors: Zheng Cao, Helyette Geman
Abstract
This article introduces a Hype-Adjusted Probability Measure in the context of a new Natural Language Processing (NLP) approach for stock return and volatility forecasting. A novel sentiment score equation is proposed to represent the impact of intraday news on forecasting next-period stock return and volatility for selected U.S. semiconductor tickers, a very vibrant industry sector. This work improves the forecast accuracy by addressing news bias, memory, and weight, and incorporating shifts in sentiment direction. More importantly, it extends the use of the remarkable tool of change of Probability Measure developed in the finance of Asset Pricing to NLP forecasting by constructing a Hype-Adjusted Probability Measure, obtained from a redistribution of the weights in the probability space, meant to correct for excessive or insufficient news.
Complexity vs Empirical Score
- Math Complexity: 6.5/10
- Empirical Rigor: 6.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper introduces a novel Hype-Adjusted Probability Measure using Radon-Nikodym derivatives and conditional expectations (high math) and presents a complete backtest with linear discriminant analysis, accuracy metrics, and data sources like LSEG/Eikon (high empirical rigor).
Research Flowchart
flowchart TD A["Research Goal:<br>Improve NLP Stock<br>Return & Volatility<br>Forecasting"] --> B["Key Methodology:<br>1. Novel Sentiment Score<br>2. Hype-Adjusted Probability<br>Measure (HAPM)"] B --> C["Data Inputs:<br>Selected U.S.<br>Semiconductor Tickers<br>Time-Series Data"] C --> D["Computational Processes:<br>Process News (Bias/Memory/Weight)<br>Calculate Sentiment Shifts<br>Redistribute Probability Weights<br>via Change of Measure"] D --> E["Key Outcomes:<br>1. Improved Forecast Accuracy<br>2. Corrects News Bias/Excess<br>3. Extends Finance Tools<br>to NLP"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper introduces a Hype-Adjusted Probability Measure for NLP-based stock return and volatility forecasting, with a novel sentiment score equation to capture intraday news impact.
Source sentence
This article introduces a Hype-Adjusted Probability Measure in the context of a new Natural Language Processing (NLP) approach for stock return and volatility forecasting.
Assumes 1
- The model assumes three states of nature (Up, Level, Down) for the underlying probability space. model
Source sentence
We suppose that there are 3 states of nature ω1, ω2, ω3 in the universe Ω, labeled as Up, Level, and Down.
Data
- LSEG and Eikon API news dataset text-news commercial
Source sentence
Primary data sources for this study were obtained from LSEG and the Eikon API [11], which provide access to over half a million news articles related to 30 semiconductor stocks.
Costs
Not applicable
Finds 5
- Validation accuracy for market return direction improves from 51.7% (baseline) to 70.0% (LDA model) and further to 78.3% with optimized scores. performance
Source sentence
In the prediction of market return direction, validation accuracy improves from 51.7% in the baseline model to 70.0% with a LDA model (as presented by Deveikyte et al [7]) using adjusted scores, and further to 78.3% with the optimized scores.
- Validation accuracy for volatility direction rises from 53.8% (baseline) to 72.1% and 75.0% in subsequent models. performance
Source sentence
Similarly, for volatility direction, accuracy increases from 53.8% in the baseline model to 72.1% and 75.0% in the corresponding models.
- The expected average difference between original volatility and the hype‑adjusted volatility is approximately –0.0068 (–0.68%). predictive-accuracy
Source sentence
Under a hype-adjusted probability measure of the selected date range, we calculate that the expected average difference between the original volatility and the adjusted one, based on the weights provided in the semiconductor dataset, is approximately −0.0068 (or −0.68%).
- Overall prediction accuracy improves by about 8% when using the hype‑adjusted sentiment scores. predictive-accuracy
Source sentence
We note however, that despite this small number, the accuracy of the prediction was improved by 8%.
- Accuracy increase of +8.3% for market return direction and +2.9% for volatility direction highlights refined precision from optimized sentiment modeling. predictive-accuracy
Source sentence
The observed accuracy increase of +8.3% for market return direction and +2.9% for volatility direction highlights the refined precision achieved through optimized sentiment modeling.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 35 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (32)
- A sentiment analysis approach to the prediction of market volatility (2022) — link
- The Pricing of Options and Corporate Liabilities (1973) — link
- Master’s Thesis: The Hyped-Adjusted Probability Measure (2025)
- LSEG Data and Analytics (2024) — link
- Introduction to News Sentiment Analysis with Eikon Data APIs - a Python example (2024) — link
- Linear Discriminant Analysis: A Brief Tutorial (1998) — link
- Financial News Data (2022) — link
- Financial Reports SEC (2022) — link
and 24 more
- EDGAR Crawler (2022) — link
- Financial News Headlines (2022) — link
- Monte Carlo Methods in Financial Engineering (2004)
- A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options (1993)
- Arbitrage-Free SVI Volatility Surfaces (2014)
- Honors Senior Thesis: Stochastic Calculus for Option Pricing with Convex Duality, Logistic Model, & Numerical Examination — → 2408.05672
- Theoretical and Empirical Validation of Heston Model — → 2409.12453
- Convex Duality in Continuous Option Pricing Models (2024) — link
- Martingales and arbitrage in multiperiod securities markets (1979) — link
- The Theory of Speculation (1900) — link
- Changes of Num' e raire, Changes of Probability Measure and Option Pricing (1995) — link
- Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (2000)
- Sentiment Analysis and Opinion Mining (2012) — link
- VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text (2014) — link
- Driven to Distraction: Extraneous Events and Underreaction to Earnings News (2009) — link
- Investor Sentiment and the Cross-Section of Stock Returns (2006) — link
- Sentiment-Guided Adversarial Learning for Stock Price Prediction (2021)
- New News is Bad News — → 2309.05560
- A Neural Probabilistic Language Model (2003)
- Twitter sentiment around the earnings announcement events (2016) — link
- Does Unusual News Forecast Market Stress? (2019)
- Measuring News Sentiment (2020) — link
- The Importance of the Forward Measure for the Pricing of Interest Rate Derivatives (1989)
- Proof That Properly Anticipated Prices Fluctuate Randomly (1965)
3 of 32 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.