Learning to Aggregate Zero-Shot LLM Agents for Corporate Disclosure Classification

Paper: arXiv 2603.20965

Authors: Kemal Kirtac

Abstract

This paper studies whether a lightweight supervised aggregator can combine diverse zero-shot large language model outputs into a stronger downstream signal for corporate disclosure classification. Zero-shot LLMs can read disclosures without task-specific fine-tuning, but their predictions often vary across prompt perspectives, model families, and confidence levels. I examine this problem with a multi-prompt framework in which three fixed zero-shot LLM classifiers read each disclosure from different financial perspectives and output a sentiment label, a confidence score, and a short rationale. A logistic meta-classifier then aggregates these outputs to predict next-day stock return direction. To reduce pretrained-model contamination, I restrict evaluation to a post-release sample of 9{,}860 U.S.\ corporate disclosures issued by large publicly traded firms between January 2025 and March 2026, after the release of the frozen base LLMs used in the experiment. Results show that the trained aggregator outperforms single classifiers, majority vote, confidence-weighted voting, a zero-shot LLM judge, and a FinBERT baseline. Balanced accuracy rises from 0.566 for the best single classifier to 0.606 for the trained aggregator. The gain is largest in mixed-signal disclosures where classifiers disagree. The results suggest that zero-shot LLM outputs contain complementary financial signals, while also showing that the strongest gains come from supervised aggregation rather than from zero-shot voting alone.

Complexity vs Empirical Score

• Math Complexity: 3.0/10
• Empirical Rigor: 7.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper relies on straightforward empirical methods (logistic regression meta-classification) rather than complex mathematics, but demonstrates strong empirical rigor through a carefully controlled out-of-sample period designed to avoid LLM data contamination. The approach of using supervised learning to aggregate zero-shot multi-prompt LLM perspectives for financial return prediction is highly practical and novel.

Research Flowchart

  flowchart TD
    A["Research Goal:<br>Can a supervised meta-classifier combine diverse zero-shot LLM agent outputs to improve corporate disclosure classification?"] --> B["Data Input:<br>9,860 U.S. corporate disclosures (Jan 2025–Mar 2026)<br><i>Post-release sample to avoid data contamination</i>"]
    
    B --> C["Multi-Prompt Zero-Shot Agents:<br>3 fixed LLM classifiers analyze each disclosure from different financial perspectives<br><i>Outputs: Sentiment Label, Confidence Score, Rationale</i>"]
    
    C --> D["Supervised Aggregation:<br>Logistic Meta-Classifier trains on agent outputs<br><i>Evaluated against Single LLMs, Voting, LLM Judge & FinBERT</i>"]
    
    D --> E["Key Findings:<br>1. Aggregator outperforms all baselines (Balanced Accuracy: 0.606 vs 0.566 best single LLM)<br>2. Largest performance gains occur in mixed-signal/disagreed disclosures"]

Cited References (34)

1. Giving Content to Investor Sentiment: The Role of Media in the Stock Market (2007) — link
2. More Than Words: Quantifying Language to Measure Firms’ Fundamentals (2008) — link
3. Earnings Conference Calls and Stock Returns: The Incremental Informativeness of Textual Tone (2012) — link
4. Tone Management (2014) — link
5. Evidence on the Information Content of Text in Analyst Reports (2014) — link
6. Annual Report Readability, Current Earnings, and Earnings Persistence (2008) — link
7. When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10- K s (2011) — link
8. In Search of Attention (2011) — link

1 of 34 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.