Paper: arXiv 2408.04948
Authors: Bhaskarjit Sarmah, Benika Hall, Rohan Rao, Sunil Patel, Stefano Pasquali, Dhagash Mehta
Abstract
Extraction and interpretation of intricate information from unstructured text data arising in financial applications, such as earnings call transcripts, present substantial challenges to large language models (LLMs) even using the current best practices to use Retrieval Augmented Generation (RAG) (referred to as VectorRAG techniques which utilize vector databases for information retrieval) due to challenges such as domain specific terminology and complex formats of the documents. We introduce a novel approach based on a combination, called HybridRAG, of the Knowledge Graphs (KGs) based RAG techniques (called GraphRAG) and VectorRAG techniques to enhance question-answer (Q&A) systems for information extraction from financial documents that is shown to be capable of generating accurate and contextually relevant answers. Using experiments on a set of financial earning call transcripts documents which come in the form of Q&A format, and hence provide a natural set of pairs of ground-truth Q&As, we show that HybridRAG which retrieves context from both vector database and KG outperforms both traditional VectorRAG and GraphRAG individually when evaluated at both the retrieval and generation stages in terms of retrieval accuracy and answer generation. The proposed technique has applications beyond the financial domain
Complexity vs Empirical Score
- Math Complexity: 2.0/10
- Empirical Rigor: 7.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper focuses on an implementation-heavy RAG methodology with empirical validation on financial datasets, but relies on established ML techniques without dense mathematical derivations.
Research Flowchart
flowchart TD
A["Research Goal: Enhance Info Extraction from Financial Text"] --> B["Input: Financial Earnings Call Transcripts"]
B --> C["HybridRAG Methodology"]
C --> D{"Parallel Retrieval"}
D --> E["VectorRAG: Vector DB Search"]
D --> F["GraphRAG: Knowledge Graph Search"]
E & F --> G["Context Synthesis & LLM Generation"]
G --> H["Outcome: HybridRAG Outperforms VectorRAG & GraphRAG"]
H --> I["Result: Accurate & Contextual Q&A"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes HybridRAG, a combination of Knowledge Graph‑based RAG (GraphRAG) and Vector‑based RAG (VectorRAG) to improve information extraction from financial documents.
Source sentence
We introduce a novel approach based on a combination, called HybridRAG, of the Knowledge Graphs (KGs) based RAG techniques (called GraphRAG) and VectorRAG techniques to enhance question-answer (Q&A) systems for information extraction from financial documents
Assumes
No explicit assumptions found in the text.
Data
- Earnings call transcripts of Nifty 50 constituents text-news proprietary
Source sentence
... Our dataset encompasses 50 transcripts for this quarter, spanning over 50 companies within Nifty 50 universe
The Graph (subgraphs)
Costs
Not applicable
Finds 3
- HybridRAG achieves the highest answer relevancy score of 0.96, outperforming VectorRAG (0.91) and GraphRAG (0.89). performance
Source sentence
Answer relevancy scores varied across the methods, with HybridRAG outperforming the others at 0.96, followed by VectorRAG at 0.91, and GraphRAG at 0.89.
- HybridRAG and GraphRAG both obtain a faithfulness score of 0.96, slightly higher than VectorRAG's 0.94. performance
Source sentence
In terms of Faithfulness, GraphRAG and HybridRAG demonstrated superior performance, both achieving a score of 0.96, while VectorRAG trailed slightly with a score of 0.94.
- HybridRAG attains perfect context recall (1.0) while maintaining high faithfulness, though its context precision is lower (0.79) compared to GraphRAG (0.96). performance
Source sentence
However, in context recall, both VectorRAG and HybridRAG achieved perfect scores of 1, while GraphRAG lagged behind at 0.85.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.
3 papers in the archive : Towards Temporal-Aware Multi-Modal Retrieval Augmented …
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.