Paper: arXiv 2510.01526

Authors: Mengyu Wang, Sotirios Sabanis, Miguel de Carvalho, Shay B. Cohen, Tiejun Ma

Abstract

Domain-specific quantitative reasoning remains a major challenge for large language models (LLMs), especially in fields requiring expert knowledge and complex question answering (QA). In this work, we propose Expert Question Decomposition (EQD), an approach designed to balance the use of domain knowledge with computational efficiency. EQD is built on a two-step fine-tuning framework and guided by a reward function that measures the effectiveness of generated sub-questions in improving QA outcomes. It requires only a few thousand training examples and a single A100 GPU for fine-tuning, with inference time comparable to zero-shot prompting. Beyond its efficiency, EQD outperforms state-of-the-art domain-tuned models and advanced prompting strategies. We evaluate EQD in the financial domain, characterized by specialized knowledge and complex quantitative reasoning, across four benchmark datasets. Our method consistently improves QA performance by 0.6% to 10.5% across different LLMs. Our analysis reveals an important insight: in domain-specific QA, a single supporting question often provides greater benefit than detailed guidance steps.

Complexity vs Empirical Score

  • Math Complexity: 1.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Philosophers — conceptual discussion, lower math and empirical scores

Why this score: The paper focuses on NLP techniques (prompting, fine-tuning) with minimal advanced mathematics, and while it reports performance metrics on benchmark datasets, it lacks code, backtests, or statistical rigor typical of empirical quant finance research.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Enhance Domain-Specific Quantitative Reasoning<br>for LLMs Efficiently"] --> B["Methodology: Expert Question Decomposition EQD"]
  B --> C{"Data & Inputs<br>Financial Domain QA Datasets<br>~3-5k Training Examples"}
  C --> D["Two-Step Fine-Tuning Framework<br>Guided by Reward Function"]
  D --> E["Computational Process<br>Train on A100 GPU<br>Inference Time ≈ Zero-Shot Prompting"]
  E --> F["Outcomes & Findings<br>Consistent Performance Gain 0.6% to 10.5%<br>Key Insight: Single Supporting Question > Detailed Steps"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes Expert Question Decomposition (EQD), a two‑step fine‑tuning framework that generates concise supporting sub‑questions to improve domain‑specific quantitative QA.

Source sentence
In this work, we propose Expert Question Decomposition (EQD), an approach designed to balance the use of domain knowledge with computational efficiency.

Assumes 1

  • A single concise supporting question is sufficient to improve QA performance more than detailed step‑by‑step guidance. other
    Source sentence
    a single supporting question often provides greater benefit than detailed guidance steps.

Data

  • ConvFinQA benchmark public
    Source sentence
    In our experiments, we use just two datasets, ConvFinQA and FinQA, among the many datasets used for financial LLM fine-tuning
  • FinQA benchmark public
    Source sentence
    In our experiments, we use just two datasets, ConvFinQA and FinQA, among the many datasets used for financial LLM fine-tuning

Costs

Not applicable The paper reports no trading or execution result.

Finds 3

  • EQD consistently improves QA performance across four financial datasets by 0.6% to 10.5% for multiple LLMs, outperforming advanced domain‑adapted models and prompting techniques. performance
    Source sentence
    Across four financial QA benchmarks, it achieves performance gains of 0.6% to 10.5% across multiple LLMs, out-performing advanced domain-adapted models and prompting techniques.
  • A single critical supporting question yields greater benefit than extensive step‑by‑step CoT prompting in domain‑specific QA. other
    Source sentence
    a single supporting question often provides greater benefit than detailed guidance steps.
  • EQD requires only a few thousand training examples and a single A100 GPU, with inference time comparable to zero‑shot prompting. performance
    Source sentence
    It requires only a few thousand training examples and a single A100 GPU for fine-tuning, with inference time comparable to zero-shot prompting.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 28 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (42)

  1. FinQAPT: Empowering Financial Decisions with End-to-End LLM-driven Question Answering Pipeline (2024)
  2. Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains
  3. GPT-4o system card (2024) — link
  4. The Llama 3 Herd of Models (2024)
  5. FinQA : A Dataset of Numerical Reasoning over Financial Data (2021)
  6. FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets — → 2310.04793
  7. Investlm: A large language model for investment using financial domain instruction tuning — → 2309.13064
  8. Improving retrieval for rag based question answering models on financial documents (2024) — link
and 34 more
  1. Medical graph rag: Towards safe medical large language model via graph retrieval-augmented generation (2024) — link
  2. Chain-of-thought prompting elicits reasoning in large language models (2022)
  3. Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
  4. PIXIU: a large language model, instruction data and evaluation benchmark for finance (2023)
  5. Bloomberggpt: A large language model for finance (2023) — link
  6. Evaluating llms’ mathematical reasoning in financial document question answering (2024)
  7. M-QALM: A Benchmark to Assess Clinical Reading Comprehension and Knowledge Recall in Large Language Models via Question Answering (2024) — link
  8. The sciqa scientific question answering benchmark for scholarly knowledge (2023)
  9. ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering (2022)
  10. TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance (2021)
  11. ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts (2022)
  12. FinGEAR: Financial Mapping-Guided Enhanced Answer Retrieval (2025) — link
  13. MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data (2022)
  14. Mana-net: Mitigating aggregated sentiment homogenization with news weighting for enhanced market prediction — → 2409.05698
  15. TaPas: Weakly Supervised Table Parsing via Pre-training (2020)
  16. FinNLP-AgentScen-2024 Shared Task: Financial Challenges in Large Language Models-FinLLMs (2024)
  17. Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications — → 2408.11878
  18. Few-shot In-context Learning on Knowledge Base Question Answering (2023)
  19. Proximal policy optimization algorithms (2017) — link
  20. Modeling News Interactions and Influence for Financial Market Prediction — → 2410.10614
  21. GPT-4 technical report (2023) — link
  22. Star: Bootstrapping reasoning with reasoning (2022)
  23. Revolutionizing Finance with LLMs: An Overview of Applications and Insights (2024)
  24. Lora: Low-rank adaptation of large language models. (2022)
  25. Decomposed Prompting: A Modular Approach for Solving Complex Tasks
  26. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions (2023)
  27. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
  28. CoMAT : Chain of mathematically annotated thought improves mathematical reasoning (2024) — link
  29. Pre-training Time Series Models with Stock Data Customization (2025)
  30. Mtmt: Consolidating multiple thinking modes to form a thought tree for strengthening llm (2024) — link
  31. A Step Closer to Comprehensive Answers: Constrained Multi-Stage Question Decomposition with Large Language Models (2023)
  32. Question decomposition tree for answering complex questions over knowledge bases (2023)
  33. ComposeRAG: A Modular and Composable RAG for Corpus-Grounded Multi-Hop Question Answering (2025) — link
  34. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (2025) — link

5 of 42 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.