Paper: arXiv 2408.04948

Authors: Bhaskarjit Sarmah, Benika Hall, Rohan Rao, Sunil Patel, Stefano Pasquali, Dhagash Mehta

Abstract

Extraction and interpretation of intricate information from unstructured text data arising in financial applications, such as earnings call transcripts, present substantial challenges to large language models (LLMs) even using the current best practices to use Retrieval Augmented Generation (RAG) (referred to as VectorRAG techniques which utilize vector databases for information retrieval) due to challenges such as domain specific terminology and complex formats of the documents. We introduce a novel approach based on a combination, called HybridRAG, of the Knowledge Graphs (KGs) based RAG techniques (called GraphRAG) and VectorRAG techniques to enhance question-answer (Q&A) systems for information extraction from financial documents that is shown to be capable of generating accurate and contextually relevant answers. Using experiments on a set of financial earning call transcripts documents which come in the form of Q&A format, and hence provide a natural set of pairs of ground-truth Q&As, we show that HybridRAG which retrieves context from both vector database and KG outperforms both traditional VectorRAG and GraphRAG individually when evaluated at both the retrieval and generation stages in terms of retrieval accuracy and answer generation. The proposed technique has applications beyond the financial domain

Complexity vs Empirical Score

  • Math Complexity: 2.0/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper focuses on an implementation-heavy RAG methodology with empirical validation on financial datasets, but relies on established ML techniques without dense mathematical derivations.

Research Flowchart

  flowchart TD
  A["Research Goal: Enhance Info Extraction from Financial Text"] --> B["Input: Financial Earnings Call Transcripts"]
  B --> C["HybridRAG Methodology"]
  C --> D{"Parallel Retrieval"}
  D --> E["VectorRAG: Vector DB Search"]
  D --> F["GraphRAG: Knowledge Graph Search"]
  E & F --> G["Context Synthesis & LLM Generation"]
  G --> H["Outcome: HybridRAG Outperforms VectorRAG & GraphRAG"]
  H --> I["Result: Accurate & Contextual Q&A"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes HybridRAG, a combination of Knowledge Graph‑based RAG (GraphRAG) and Vector‑based RAG (VectorRAG) to improve information extraction from financial documents.

Source sentence
We introduce a novel approach based on a combination, called HybridRAG, of the Knowledge Graphs (KGs) based RAG techniques (called GraphRAG) and VectorRAG techniques to enhance question-answer (Q&A) systems for information extraction from financial documents

Assumes

No explicit assumptions found in the text.

Data

  • Earnings call transcripts of Nifty 50 constituents · company websites (NIFTY 50 universe) text-news proprietary
    quarter ending June 2023 (Q1 FY2024) · Nifty 50 companies
    Source sentence
    ... Our dataset encompasses 50 transcripts for this quarter, spanning over 50 companies within Nifty 50 universe

Recognised sources: The Graph (subgraphs)

Costs

Not applicable The paper reports no trading or execution result.

Finds 3

  • HybridRAG achieves the highest answer relevancy score of 0.96, outperforming VectorRAG (0.91) and GraphRAG (0.89). performance
    Source sentence
    Answer relevancy scores varied across the methods, with HybridRAG outperforming the others at 0.96, followed by VectorRAG at 0.91, and GraphRAG at 0.89.
  • HybridRAG and GraphRAG both obtain a faithfulness score of 0.96, slightly higher than VectorRAG's 0.94. performance
    Source sentence
    In terms of Faithfulness, GraphRAG and HybridRAG demonstrated superior performance, both achieving a score of 0.96, while VectorRAG trailed slightly with a score of 0.94.
  • HybridRAG attains perfect context recall (1.0) while maintaining high faithfulness, though its context precision is lower (0.79) compared to GraphRAG (0.96). performance
    Source sentence
    However, in context recall, both VectorRAG and HybridRAG achieved perfect scores of 1, while GraphRAG lagged behind at 0.85.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 3 papers in the archive (3 by independent authors): Towards Temporal-Aware Multi-Modal Retrieval Augmented … (uses its method or data)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.