Paper: arXiv 2603.20252

Authors: Mahesh Kumar, Bhaskarjit Sarmah, Stefano Pasquali

Abstract

As organizations increasingly integrate AI-powered question-answering systems into financial information systems for compliance, risk assessment, and decision support, ensuring the factual accuracy of AI-generated outputs becomes a critical engineering challenge. Current Knowledge Graph (KG)-augmented QA systems lack systematic mechanisms to detect hallucinations - factually incorrect outputs that undermine reliability and user trust. We introduce FinBench-QA-Hallucination, a benchmark for evaluating hallucination detection methods in KG-augmented financial QA over SEC 10-K filings. The dataset contains 755 annotated examples from 300 pages, each labeled for groundedness using a conservative evidence-linkage protocol requiring support from both textual chunks and extracted relational triplets. We evaluate six detection approaches - LLM judges, fine-tuned classifiers, Natural Language Inference (NLI) models, span detectors, and embedding-based methods under two conditions: with and without KG triplets. Results show that LLM-based judges and embedding approaches achieve the highest performance (F1: 0.82-0.86) under clean conditions. However, most methods degrade significantly when noisy triplets are introduced, with Matthews Correlation Coefficient (MCC) dropping 44-84 percent, while embedding methods remain relatively robust with only 9 percent degradation. Statistical tests (Cochran’s Q and McNemar) confirm significant performance differences (p < 0.001). Our findings highlight vulnerabilities in current KG-augmented systems and provide insights for building reliable financial information systems, where hallucinations can lead to regulatory violations and flawed decisions. The benchmark also offers a framework for integrating AI reliability evaluation into information system design across other high-stakes domains such as healthcare, legal, and government.

Complexity vs Empirical Score

  • Math Complexity: 2.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: no out-of-sample test; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper presents an empirically rigorous benchmark (FinBench-QA-Hallucination) evaluating hallucination detection in GraphRAG systems using SEC 10-K filings. While it features minimal mathematical derivation, it excels in statistical testing (Cochran’s Q, McNemar), robust noise-sensitivity analysis, and practical insights for financial NLP systems.

Research Flowchart

  flowchart TD
    A["<b>Research Goal</b><br/>Benchmark & evaluate hallucination detection in GraphRAG financial QA"] --> B["<b>Dataset Construction</b><br/>SEC 10-K Filings (300 pages)"]
    B --> C["<b>FinBench-QA-Hallucination</b><br/>755 annotated examples labeled via conservative evidence-linkage"]
    C --> D["<b>Evaluation Protocol</b><br/>6 detection methods tested under Clean vs. Noisy KG Triplet conditions"]
    D --> E["<b>Key Findings</b><br/>• LLM judges & embeddings perform best (F1: 0.82-0.86)<br/>• Noisy triplets degrade most methods (MCC drops 44-84%)<br/>• Embedding methods remain robust (only 9% drop)"]

Cited References (31)

  1. The dawn after the dark: An empirical study on factuality hallucination in large language models (2024) — link
  2. Helma: A large-scale hallucination evaluation benchmark for large language models (2023) — link
  3. RefChecker: Reference-based fine-grained hallucination checker and benchmark for large language models (2024) — link
  4. Factchd: Benchmarking fact-conflicting hallucination detection (2023) — link
  5. Anah-v2: Scaling analytical hallucination annotation of large language models (2024)
  6. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models (2023) — link
  7. Unsupervised real-time hallucination detection based on the internal states of large language models (2024) — link
  8. The Geometry of Truth: Layer-wise Semantic Dynamics for Hallucination Detection in Large Language Models (2025) — link
and 23 more
  1. Medhallu: A comprehensive benchmark for detecting medical hallucinations in large language models (2025) — link
  2. Gnn-rag: Graph neural retrieval for large language model reasoning (2024) — link
  3. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering (2024)
  4. Think-on-graph 2.0: Deep and faithful large language model reasoning with knowledge-guided retrieval augmented generation (2024) — link
  5. Finbert: Financial sentiment analysis with pre-trained language models (2019) — link
  6. Bloomberggpt: A large language model for finance (2023) — link
  7. Investlm: A large language model for investment using financial domain instruction tuning — → 2309.13064
  8. PIXIU: A Comprehensive Benchmark, Instruction Dataset and Large Language Model for Finance (2023) — link
  9. Finben: A holistic financial benchmark for large language models (2024)
  10. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation (2020) — link
  11. The receiver operating characteristic curve accurately assesses imbalanced datasets (2024) — link
  12. A simple sequentially rejective multiple test procedure (1979)
  13. Approximate statistical tests for comparing supervised classification learning algorithms (1998) — link
  14. Note on the sampling error of the difference between correlated proportions or percentages (1947) — link
  15. FinReflectKG: Agentic Construction and Evaluation of Financial Knowledge Graphs (2025) — link
  16. Qwen3-235B-A22B-Instruct-2507
  17. GPT-OSS-120B
  18. Llama-3-Patronus-Lynx-8B-Instruct
  19. DeBERTa-v3-large-mnli-fever-anli-ling-wanli
  20. Vectara Hallucination Evaluation Model
  21. LettuceDetect: ModernBERT Hallucination Detector
  22. Qwen3-Embedding-0.6B
  23. Stella Embedding Model 400M v5

1 of 31 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.