Paper: arXiv 2611.02258

Authors: Xiongyu Chen, Sizhen Zhu, Zhuoer Wang

Abstract

Large language models are increasingly used to answer questions over financial filings, where an unfaithful answer can mislead investment, compliance, or client decisions. Hallucination detectors are a common safeguard, but they are usually evaluated with a single binary score that reveals neither which kinds of errors analysesed nor why. This paper studies hallucination detection on PHANTOM, a long-context benchmark built from SEC filings, at the level of hallucination type. We first analyze 15,443 paired hallucinated and faithful answers without any detector and find that hallucinated answers are more than twice as long as faithful ones, and that lexical grounding separates the two classes only because faithful answers are bimodal. We then generate hallucinations in nine financially motivated categories, confirm each with a separate verification pass, and diagnose an LLM-based detector on 4,086 usable examples. Generation success varies from 66.3% to 100% across categories. The detector reaches 80.2% binary accuracy but only 44.7% subtype accuracy, never assigns two of the nine categories, and misses between 5.3% and 46.8% of hallucinations depending on category; these misses are unrelated to input length but rise with how far an answer departs from the correct one. Guided by this diagnosis, we propose a two-stage hierarchical detector that first decides whether an answer is hallucinated and then assigns a subtype, raising binary accuracy to 95.6% and subtype accuracy to 56.1%. The results show that type-level evaluation exposes detector weaknesses that aggregate metrics conceal.

Complexity vs Empirical Score

  • Math Complexity: 3.0/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: no out-of-sample test; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper presents a robust empirical analysis of hallucination detection in financial LLMs, backed by a detailed dataset analysis and a proposed hierarchical detector. While the mathematical complexity is moderate, the empirical rigor and novel approach to type-aware evaluation are strong.

Research Flowchart

  flowchart TD
    A[Research Goal: Improve Hallucination Detection in Financial QA] --> B{Key Methodology Steps};

    B --> C[Data/Inputs: PHANTOM benchmark (SEC filings), 15,443 paired answers, 4,086 generated hallucinations (9 types)];
    C --> D[Computational Processes: LLM-based Detector Diagnosis; Hierarchical Detector Development];

    D --> E[Key Findings:
        - Hallucinations are longer than faithful answers.
        - Binary detector accuracy: 80.2%, Subtype accuracy: 44.7%.
        - Detector misses 5.3-46.8% of hallucinations.
        - Hierarchical detector raises binary accuracy to 95.6%, subtype accuracy to 56.1%.
        - Type-level evaluation reveals detector weaknesses.
    ];