Paper: arXiv 2602.12490

Authors: Junyu Chen, Tom Boot, Lingwei Kong, Weining Wang

Abstract

Conditional Value-at-Risk (CoVaR) quantifies systemic financial risk by measuring the loss quantile of one asset, conditional on another asset experiencing distress. We develop a Transformer-based methodology that integrates financial news articles directly with market data to improve CoVaR estimates. Unlike approaches that use predefined sentiment scores, our method incorporates raw text embeddings generated by a large language model (LLM). We prove explicit error bounds for our Transformer CoVaR estimator, showing that accurate CoVaR learning is possible even with small datasets. Using U.S. market returns and Reuters news items from 2006–2013, our out-of-sample results show that textual information impacts the CoVaR forecasts. With better predictive performance, we identify a pronounced negative dip during market stress periods across several equity assets when comparing the Transformer-based CoVaR to both the CoVaR without text and the CoVaR using traditional sentiment measures. Our results show that textual data can be used to effectively model systemic risk without requiring prohibitively large data sets.

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a strong theoretical framework establishing explicit error bounds and convergence rates for Transformer-based quantile regressions in high dimensions. The empirical evaluation is rigorous, demonstrating clear out-of-sample improvements over traditional sentiment approaches when estimating systemic risk (CoVaR) during major crisis periods.

Research Flowchart

  flowchart TD
    A[Research Goal: Improve Systemic Risk<br>CoVaR Estimation via Financial News] --> B[Data Inputs: Reuters News Articles 2006–2013<br>& US Market Return Series]
    B --> C[Extract Raw Text Embeddings<br>Using Large Language Models]
    C --> D[Compute Transformer-Based CoVaR<br>Combining Text & Market Data]
    D --> E[Out-of-Sample Validation &<br>Theoretical Error Bound Proofs]
    E --> F[Findings: Superior Predictive Accuracy &<br>Deeper Risk Dip Captures During Stress]

Cited References (98)

  1. Stability Theorems for Maxima of Dependent Sequences (1992)
  2. The world price of sentiment risk (2017) — link
  3. Financial Contagion (2000)
  4. Systemic Risk and Stability in Financial Networks (2015)
  5. Econometric Measures of Connectedness and Systemic Risk in the Finance and Insurance Sectors (2012)
  6. On the Network Topology of Variance Decompositions: Measuring the Connectedness of Financial Firms (2014)
  7. Complexity, Concentration and Contagion (2011)
  8. Vulnerable Banks (2015)
and 90 more
  1. Measuring Systemic Risk (2017)
  2. Banking Stability Measures (2009)
  3. C o V a R (2016) — link
  4. Financial Network Systemic Risk Contributions (2014) — link
  5. TENET : Tail-Event driven NETwork risk (2016) — link
  6. Modeling systemic risk with Markov Switching Graphical SUR models (2019) — link
  7. Modelling Systemic Risk Using Neural Network Quantile Regression (2022) — link
  8. Expected Shortfall LASSO (2024) — link
  9. The Structure of Economic News (2020) — link
  10. Nowcasting GDP using tone-adjusted time varying news topics: Evidence from the financial press (2023)
  11. Forecasting macroeconomic tail risk in real time: Do textual data add value? (2025) — link
  12. Irrational Exuberance (2000) — link
  13. Giving Content to Investor Sentiment: The Role of Media in the Stock Market (2007) — link
  14. Sentiment during Recessions (2013) — link
  15. Beyond valence: Toward a model of emotion-specific influences on judgement and choice (2000) — link
  16. Fear, anger, and risk (2001)
  17. When Is a Liability Not a Liability? T extual Analysis, Dictionaries, and 10-Ks (2011) — link
  18. Predicting Returns With Text Data (2019) — link
  19. Does sentiment help in asset pricing? A novel approach using large language models and market-based labels (2024)
  20. MANA - N et: Mitigating Aggregated Sentiment Homogenization with News Weighting for Enhanced Market Prediction (2024) — link
  21. Does Finance Benefit Society? A Language Embedding Approach (2025) — link
  22. The Causal Impact of Media in Financial Markets (2011) — link
  23. Journalists and the Stock Market (2012)
  24. Front-Page News: The Effect of News Positioning on Financial Markets (2024) — link
  25. Learning Fundamentals from Text (2024) — link
  26. Measuring Readability in Financial Disclosures (2014) — link
  27. Predicting risk from financial reports with regression (2009)
  28. The value of news for economic developments (2019) — link
  29. Regression Quantiles (1978)
  30. Nonparametric regression using deep neural networks with R e LU activation function (2020)
  31. Adaptivity of deep R e LU network for learning in B esov and mixed smooth B esov spaces: Optimal rate and curse of dimensionality (2019)
  32. Attention is All You Need (2017)
  33. Investigating the Role of Feed-Forward Networks in Transformers Using Parallel Attention and Feed-Forward Net Design (2023) — link
  34. Are Transformers universal approximators of sequence-to-sequence functions? (2020) — link
  35. Attention is not only a weight: Analyzing Transformers with vector norms (2021)
  36. When Do T ransformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective (2025) — link
  37. Comparative Study of CNN and RNN for Natural Language Processing (2017) — link
  38. Transformers and Their Roles as Time Series Foundation Models (2025) — link
  39. On the Rate of Convergence of a Classifier Based on a Transformer Encoder (2022) — link
  40. On the rate of convergence of an over-parametrized Transformer classifier learned by gradient descent (2024) — link
  41. RiskLabs: Predicting Financial Risk Using Large Language Model based on Multimodal and Multi-Sources Data (2025) — link
  42. HTML : H ierarchical T ransformer-based M ulti-task L earning for Volatility Prediction (2020) — link
  43. Pre B it — A multimodal model with Twitter F in BERT embeddings for extreme price movement prediction of B itcoin (2023) — link
  44. StockTime: A Time Series Specialized Large Language Model Architecture for Stock Price Prediction (2024) — link
  45. News N et- SDF : Stochastic Discount Factor Estimation with Pretrained Language Model News Embeddings via Adversarial Networks (2025) — link
  46. Higher Order Transformers: Enhancing Stock Movement Prediction On Multimodal Time-Series Data (2024) — link
  47. MAEC : A Multimodal Aligned Earnings Conference Call Dataset for Financial Risk Prediction (2020) — link
  48. Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection (2023) — link
  49. In-Context Learning with Representations: Contextual Generalization of Trained Transformers (2024) — link
  50. Asymptotic theory of in-context learning by linear attention (2025) — link
  51. Transformers are Universal In-context Learners (2024) — link
  52. Transformers are Minimax Optimal Nonparametric In-Context Learners (2024) — link
  53. Language Models are Few-Shot Learners (2020) — link
  54. Incorporating volatility updating into the historical simulation method for V a R (1998) — link
  55. Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of U nited K ingdom Inflation (1982) — link
  56. Generalized autoregressive conditional heteroskedasticity (1986) — link
  57. Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach (2000) — link
  58. Quantile Regression in Risk Calibration (2015) — link
  59. Approximation Bounds for Transformer Networks with Application to Regression (2025) — link
  60. Inductive Biases and Variable Creation in Self-Attention Mechanisms (2022) — link
  61. Sequence Length Independent Norm-Based Generalization Bounds for Transformers (2024)
  62. Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data (2024) — link
  63. Nearly-tight VC -dimension and Pseudodimension Bounds for Piecewise Linear Neural Networks (2019) — link
  64. Local Rademacher Complexities (2005)
  65. Limiting Distributions for L 1 Regression Estimators under General Conditions (1998)
  66. On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces (2020) — link
  67. Quantile regression with R e LU Networks: Estimators and minimax rates (2022) — link
  68. Quantile Additive Trend Filtering (2025) — link
  69. Covering Number Bounds of Certain Regularized Linear Function Classes (2002) — link
  70. In the beginning was the Word: LLM - V a R and LLM - ES (2026) — link
  71. Time Series Forecasting with Transformer Models and Application to Asset Management (2023) — link
  72. Using Structured Events to Predict Stock Price Movement: An Empirical Investigation (2014) — link
  73. A Survey of Large Language Models (2025) — link
  74. Gemini Embedding: Generalizable Embeddings from G emini (2025) — link
  75. Matryoshka Representation Learning (2024) — link
  76. Text mining for market prediction: A systematic review (2014) — link
  77. Deep Residual Learning for Image Recognition (2016) — link
  78. Layer Normalization (2016) — link
  79. Visualizing Data using t- SNE (2008) — link
  80. Theoretical Foundations of t- SNE for Visualizing High-Dimensional Clustered Data (2022) — link
  81. Global Business Networks (2025) — link
  82. Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis (2023) — link
  83. Gecko: Versatile Text Embeddings Distilled from Large Language Models (2024) — link
  84. What can transformers learn in-context? A case study of simple function classes (2022)
  85. How Do Nonlinear Transformers Learn and Generalize in In-Context Learning? (2024) — link
  86. Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful (2025) — link
  87. Lipschitz Widths (2023) — link
  88. Dynamic semiparametric models for expected shortfall (and V alue-at- R isk) (2019) — link
  89. Dynamic C o V a R Modeling and Estimation (2025) — link
  90. On the relationship between self-attention and convolution (2020)

0 of 98 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.