Paper: arXiv 2412.07587

Abstract

This article introduces a Hype-Adjusted Probability Measure in the context of a new Natural Language Processing (NLP) approach for stock return and volatility forecasting. A novel sentiment score equation is proposed to represent the impact of intraday news on forecasting next-period stock return and volatility for selected U.S. semiconductor tickers, a very vibrant industry sector. This work improves the forecast accuracy by addressing news bias, memory, and weight, and incorporating shifts in sentiment direction. More importantly, it extends the use of the remarkable tool of change of Probability Measure developed in the finance of Asset Pricing to NLP forecasting by constructing a Hype-Adjusted Probability Measure, obtained from a redistribution of the weights in the probability space, meant to correct for excessive or insufficient news.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces a novel Hype-Adjusted Probability Measure using Radon-Nikodym derivatives and conditional expectations (high math) and presents a complete backtest with linear discriminant analysis, accuracy metrics, and data sources like LSEG/Eikon (high empirical rigor).

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Improve NLP Stock<br>Return & Volatility<br>Forecasting"] --> B["Key Methodology:<br>1. Novel Sentiment Score<br>2. Hype-Adjusted Probability<br>Measure (HAPM)"]
  B --> C["Data Inputs:<br>Selected U.S.<br>Semiconductor Tickers<br>Time-Series Data"]
  C --> D["Computational Processes:<br>Process News (Bias/Memory/Weight)<br>Calculate Sentiment Shifts<br>Redistribute Probability Weights<br>via Change of Measure"]
  D --> E["Key Outcomes:<br>1. Improved Forecast Accuracy<br>2. Corrects News Bias/Excess<br>3. Extends Finance Tools<br>to NLP"]