Degree of Irrationality: Sentiment and Implied Volatility Surface

Paper: arXiv 2405.11730

Authors: Jiahao Weng, Yan Xie

Abstract

In this study, we constructed daily high-frequency sentiment data and used the VAR method to attempt to predict the next day’s implied volatility surface. We utilized 630,000 text data entries from the East Money Stock Forum from 2014 to 2023 and employed deep learning methods such as BERT and LSTM to build daily market sentiment indicators. By applying FFT and EMD methods for sentiment decomposition, we found that high-frequency sentiment had a stronger correlation with at-the-money (ATM) options’ implied volatility, while low-frequency sentiment was more strongly correlated with deep out-of-the-money (DOTM) options’ implied volatility. Further analysis revealed that the shape of the implied volatility surface contains richer market sentiment information beyond just market panic. We demonstrated that incorporating this sentiment information can improve the accuracy of implied volatility surface predictions.

Complexity vs Empirical Score

• Math Complexity: 5.5/10
• Empirical Rigor: 7.5/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents an innovative empirical framework linking sentiment frequency decomposition (via FFT and EMD) with the implied volatility surface using large-scale forum text and deep learning models (BERT, LSTM). It combines solid empirical rigor across a 2014-2023 dataset with clear interpretability, improving vector autoregression (VAR) surface predictions.

Research Flowchart

  flowchart TD
    A["Research Goal: Predict Implied Volatility Surface using Sentiment"] --> B["Data Collection: 630k East Money Forum Texts 2014-2023"]
    B --> C["Sentiment Indicator Construction via BERT & LSTM"]
    C --> D["Signal Decomposition using FFT & EMD"]
    D --> E["Econometric Modeling via VAR"]
    E --> F["Key Findings: High-freq sentiment correlates with ATM IV; Low-freq with DOTM IV"]
    E --> G["Outcome: Sentiment improves next-day Implied Volatility Surface prediction"]