Paper: arXiv 2505.14420
Authors: Huopu Zhang, Yanguang Liu, Miao Zhang, Zirui He, Mengnan Du
Abstract
Predicting earnings surprises from financial documents, such as earnings conference calls, regulatory filings, and financial news, has become increasingly important in financial economics. However, these financial documents present significant analytical challenges, typically containing over 5,000 words with substantial redundancy and industry-specific terminology that creates obstacles for language models. In this work, we propose the SAE-FiRE (Sparse Autoencoder for Financial Representation Enhancement) framework to address these limitations by extracting key information while eliminating redundancy. SAE-FiRE employs Sparse Autoencoders (SAEs) to decompose dense neural representations from large language models into interpretable sparse components, then applies statistical feature selection methods, including ANOVA F-tests and tree-based importance scoring, to identify the top-k most discriminative dimensions for classification. By systematically filtering out noise that might otherwise lead to overfitting, we enable more robust and generalizable predictions. Experimental results across three financial datasets demonstrate that SAE-FiRE significantly outperforms baseline approaches.
Complexity vs Empirical Score
- Math Complexity: 7.0/10
- Empirical Rigor: 6.5/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced neural network architectures and mathematical formulations (e.g., sparse autoencoders, gradient boosting), placing it in the high-math category. Its empirical rigor is solid, with evaluation on three real financial datasets and cross-validation, though it lacks explicit code or live backtesting details.
Research Flowchart
flowchart TD A["Research Goal: Predict Earnings Surprises from Financial Documents<br>(Conference Calls, Filings, News)"] --> B["Input Data<br>Textual Datasets"] B --> C["Core Methodology: SAE-FiRE Framework"] C --> D["1. Representation Extraction<br>Use LLM to generate dense text embeddings"] D --> E["2. Sparse Decomposition<br>Apply Sparse Autoencoder to get interpretable features"] E --> F["3. Feature Selection<br>ANOVA F-tests & Tree-based importance scoring<br>Select Top-k discriminative features"] F --> G["Outcomes & Findings"] G --> H["Robust Predictions<br>Reduced overfitting via noise filtering"] G --> I["Performance Boost<br>Significantly outperforms baseline approaches"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes the SAE‑FiRE framework, a sparse autoencoder‑based feature selection method for earnings surprise prediction from financial documents.
Source sentence
We propose the SAE‑FiRE framework for earnings surprise prediction that effectively suppresses feature noise in financial documents.
Assumes 5
- Earnings surprise direction is predicted as a binary supervised learning task using only the textual content of the most recent financial documents. data
Source sentence
We frame the prediction of the direction of the next earnings surprise (ES) as a supervised learning task using only the textual content of the most recent financial documents.
- Financial documents are truncated to at most 20,000 tokens for activation extraction. execution
Source sentence
For every financial document (truncated to T = 20,000 tokens), we extract each token’s residual‑stream activations
- All experiments use a fixed random seed of 42 for reproducibility. execution
Source sentence
all random seeds were fixed to 42 for reproducibility.
- Baseline models (logistic regression, XGBoost, MLP) are trained with standard hyper‑parameters as implemented in scikit‑learn and PyTorch. model
Source sentence
regression and XGBoost baselines are trained with scikit‑learn and XGBoost, while MLPs employ Py‑Torch with three hidden layers of size 256 and dropout of 0.5 and ReLU activations.
Show 1 more
- The binary classification setup simplifies the nuanced nature of earnings surprises that vary in magnitude. statistical
Source sentence
our experimental setup focuses on binary classification, which simplifies the nuanced nature of earnings surprises that often vary in magnitude and impact.
Data
- Conference Call transcripts text-news unknown
Source sentence
We employ three main datasets in our analysis: (i) Conference Call, (ii) 10‑Q Report, and (iii) FNSPID Nasdaq News.
- 10‑Q Report text-news unknown
Source sentence
We employ three main datasets in our analysis: (i) Conference Call, (ii) 10‑Q Report, and (iii) FNSPID Nasdaq News.
- FNSPID Nasdaq News text-news unknown
Source sentence
We employ three main datasets in our analysis: (i) Conference Call, (ii) 10‑Q Report, and (iii) FNSPID Nasdaq News.
Costs
Not applicable
Finds 3
- SAE‑FiRE models achieve the highest F1‑scores and AUC values across all three datasets, e.g., on Conference Calls F1‑scores of 0.759, 0.757, 0.743 and AUC of 0.673, 0.668, 0.657. performance
Source sentence
On Conference Calls, the three SAE‑FiRE models achieve the highest F1‑scores (0.759, 0.757, 0.743) and AUC values (0.673, 0.668, 0.657).
- Using top‑k selected SAE features improves performance versus using all 16K features (e.g., 0.743 F1 and 0.657 AUC vs 0.737 F1 and 0.642 AUC with all features). predictive-accuracy
Source sentence
SAE‑FiRE with selective top‑k feature retention achieves 0.743 F1 (0.657 AUC) on Conference Calls using the Gemma 2‑2B variant.
- Larger LLM (Gemma 2‑9B 131K) yields better metrics than smaller (Gemma 2‑2B 16K): accuracy 0.801 vs 0.793, AUC 0.668 vs 0.657, weighted F1 0.757 vs 0.743. performance
Source sentence
the model using 9B 131K SAE bringing accuracy from 0.793 to 0.801, AUC from 0.657 to 0.668, and weighted average F1 from 0.743 to 0.757.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 31 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (73)
- The Theory of Parsing, Translation and Compiling (1972)
- Publications Manual (1983)
- Alternation (1981) — link
- Scalable training of L1 -regularized log-linear models (2007)
- Algorithms on Strings, Trees and Sequences (1997)
- Yara Parser: A Fast and Accurate Dependency Parser (2015) — link
- Detecting deceptive discussions in conference calls (2012)
- What you say and how you say it matters: Predicting stock volatility using verbal and vocal cues (2019)
and 65 more
- Linguistic complexity in firm disclosures: Obfuscation or information? (2018)
- Legal-bert: The muppets straight out of law school (2020) — link
- Unsupervised cross-lingual representation learning at scale (2020)
- Transformer-XL: Attentive language models beyond a fixed-length context (2019)
- The extreme future stock returns following I/B/E/S earnings surprises (2006)
- Using natural language processing to assess text usefulness to readers: The case of conference calls and earnings prediction (2018) — link
- Unified pretraining framework for document understanding (2021)
- Don’t stop pretraining: Adapt language models to domains and tasks (2020)
- Unsupervised domain adaptation of contextualized embeddings for sequence labeling (2019)
- FinBERT: A large language model for extracting information from financial text (2023)
- On the market reaction to revenue and earnings surprises (2009)
- Predicting returns with text data (2019) — link
- Standardized unexpected earnings–1971-77 (1979)
- BioBERT: A pre-trained biomedical language representation model for biomedical text mining (2020)
- The information content of forward-looking statements in corporate filings: A naïve Bayesian machine learning approach (2010)
- RoBERTa: A robustly optimized BERT pretraining approach (2019) — link
- Learning word vectors for sentiment analysis (2011)
- Phenotyping of clinical notes with improved document classification models using contextualized neural language models (2019) — link
- Hierarchical transformers for long document classification (2019)
- GloVe: Global vectors for word representation (2014)
- Investigating efficiently extending transformers for long input summarization (2022) — link
- Earnings conference calls and stock returns: The incremental informativeness of textual tone (2012)
- What you say and how you say it matters: Predicting stock volatility using verbal and vocal cues (2019)
- “Why should I trust you?”: Explaining the predictions of any classifier (2016)
- Term-weighting approaches in automatic text retrieval (1988)
- Predicting corporate risk by jointly modeling company networks and dialogues in earnings conference calls (2022) — link
- Voltage: Volatility forecasting via text-audio fusion with graph convolution networks for earnings calls (2020)
- Multimodal multi-task financial risk forecasting (2020)
- Attention is all you need (2017)
- Interpreting tf-idf term weights as making relevance decisions (2008)
- NumHTML: Numeric-oriented hierarchical transformer model for multi-task financial forecasting (2022) — link
- HTML: Hierarchical transformer-based multi-task learning for volatility prediction (2020)
- Beyond 512 tokens: Siamese multi-depth transformer-based hierarchical encoder for long-form document matching (2020)
- Hierarchical attention networks for document classification (2016)
- Big Bird: Transformers for longer sequences (2020)
- HIBERT: Document level pre-training of hierarchical bidirectional transformers for document summarization (2019)
- Forecasting earnings surprises from conference call transcripts (2023)
- Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2 (2024) — link
- Longformer: The Long-Document Transformer (2020) — link
- SAELens (2024)
- Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders (2024) — link
- Representation engineering: A top-down approach to ai transparency (2023) — link
- Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders (2024) — link
- Neuronpedia: Interactive Reference and Tooling for Analyzing Neural Networks (2023) — link
- A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models (2025)
- Patterns in Unexpected Earnings as an Explanation for Post - Announcement Drift (1992) — link
- Retail Investors’ Contrarian Behavior Around News, Attention, and the Momentum Effect (2022) — link
- Conference calls and information asymmetry (2004)
- Do analysts and investors efficiently respond to managerial linguistic complexity during conference calls? (2024)
- Financial Forecasting from Textual and Tabular Time Series (2024)
- Liability or asset? The role of language in financial reports (2011)
- Macroeconomic expectations, uncertainty, and capital investment (2018)
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019)
- Longformer: The Long-Document Transformer (2020) — link
- Reformer: The Efficient Transformer (2020)
- A Five-Factor Asset Pricing Model (2015)
- Linguistic Complexity in Earnings Conference Calls and Stock Market Reactions (2018)
- Corporate Culture: Evidence from the Field (2020)
- Sparse Autoencoder (2011)
- K-Sparse Autoencoders (2014)
- Sparse coding with an overcomplete basis set: A strategy employed by V1? (1997)
- Scaling and evaluating sparse autoencoders (2025)
- Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models (2025)
- SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models (2025)
- FNSPID: A Comprehensive Financial News Dataset in Time Series — → 2402.06698
1 of 73 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.