Paper: arXiv 2510.02906

Authors: Abhinav Arun, Reetu Raj Harsh, Bhaskarjit Sarmah, Stefano Pasquali

Abstract

Multi-hop reasoning over financial disclosures is often a retrieval problem before it becomes a reasoning or generation problem: relevant facts are dispersed across sections, filings, companies, and years, and LLMs often expend excessive tokens navigating noisy context. Without precise Knowledge Graph (KG)-guided selection of relevant context, even strong reasoning models either fail to answer or consume excessive tokens, whereas KG-linked evidence enables models to focus their reasoning on composing already retrieved facts. We present FinReflectKG - MultiHop, a benchmark built on FinReflectKG, a temporally indexed financial KG that links audited triples to source chunks from S&P 100 filings (2022-2024). Mining frequent 2-3 hop subgraph patterns across sectors (via GICS taxonomy), we generate financial analyst style questions with exact supporting evidence from the KG. A two-phase pipeline first creates QA pairs via pattern-specific prompts, followed by a multi-criteria quality control evaluation to ensure QA validity. We then evaluate three controlled retrieval scenarios: (S1) precise KG-linked paths; (S2) text-only page windows centered on relevant text spans; and (S3) relevant page windows with randomizations and distractors. Across both reasoning and non-reasoning models, KG-guided precise retrieval yields substantial gains on the FinReflectKG - MultiHop QA benchmark dataset, boosting correctness scores by approximately 24 percent while reducing token utilization by approximately 84.5 percent compared to the page window setting, which reflects the traditional vector retrieval paradigm. Spanning intra-document, inter-year, and cross-company scopes, our work underscores the pivotal role of knowledge graphs in efficiently connecting evidence for multi-hop financial QA. We also release a curated subset of the benchmark (555 QA Pairs) to catalyze further research.

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: no out-of-sample test; see penalty flags)
  • Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 8.0)

Why this score: The paper focuses on practical benchmarking and retrieval implementation with a dataset, showing strong empirical rigor through backtest-style evaluation, while using relatively simple mathematical concepts like graph traversals and metrics without complex derivations.

Research Flowchart

  flowchart TD
  A["Research Goal: Evaluate KG-guided retrieval for multi-hop financial QA vs. traditional text retrieval"]
  
  B["Data Input: FinReflectKG KG<br>Source: S&P 100 filings (2022-2024)<br>Structure: Temporally indexed triples"]
  
  C["Pattern Mining: Extract frequent 2-3 hop subgraph patterns<br>via GICS taxonomy & financial disclosures"]
  
  D["Benchmark Generation: QA pairs via<br>pattern-specific prompts + quality control"]
  
  E["Computational Pipeline: Three Retrieval Scenarios<br>S1: KG-linked evidence<br>S2: Text-only page windows<br>S3: Distractors + randomization"]
  
  F["Evaluation: Reasoning & non-reasoning models<br>on FinReflectKG-MultiHop benchmark"]
  
  G["Key Findings: +24% correctness, -84.5% tokens<br>KG-guided retrieval outperforms text-only<br>intra-document, inter-year, cross-company"]
  
  A --> B
  B --> C
  C --> D
  D --> E
  E --> F
  F --> G

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper introduces FinReflectKG‑MultiHop, a multi‑hop financial QA benchmark that links a temporally indexed knowledge graph to S&P 100 10‑K filings and provides controlled retrieval settings.

Source sentence
We present FinReflectKG - MultiHop, a benchmark built on FinReflectKG, a temporally indexed financial KG that links audited triples to source chunks from S&P 100 filings (2022-2024).

Assumes

No explicit assumptions found in the text.

Data

  • FinReflectKG - MultiHop QA Benchmark · SEC 10‑K filings of S&P 100 companies benchmark public
    2022-2024 · S&P 100 constituents
    Source sentence
    FinReflectKG - MultiHop, a benchmark built on FinReflectKG, a temporally indexed financial KG that links audited triples to source chunks from S&P 100 filings (2022-2024).

Costs

Not applicable The paper reports no trading or execution result.

Finds 2

  • KG‑guided precise retrieval improves correctness scores by approximately 24 % compared to text‑only page‑window retrieval. performance
    Source sentence
    KG‑guided precise retrieval yields substantial gains on the FinReflectKG - MultiHop QA benchmark dataset, boosting correctness scores by ∼24% while reducing token utilization by ∼84.5% compared to the page‑window setting.
  • KG‑guided retrieval reduces token usage by roughly 84.5 % relative to the traditional vector‑retrieval page‑window baseline. performance
    Source sentence
    KG‑guided precise retrieval yields substantial gains on the FinReflectKG - MultiHop QA benchmark dataset, boosting correctness scores by ∼24% while reducing token utilization by ∼84.5% compared to the page‑window setting.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 8 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.