Paper: arXiv 2510.01887
Authors: Avinash Kumar Singh, Bhaskarjit Sarmah, Stefano Pasquali
Abstract
Text-to-SQL, the task of translating natural language questions into SQL queries, has long been a central challenge in NLP. While progress has been significant, applying it to the financial domain remains especially difficult due to complex schema, domain-specific terminology, and high stakes of error. Despite this, there is no dedicated large-scale financial dataset to advance research, creating a critical gap. To address this, we introduce a curated financial dataset (FINCH) comprising 292 tables and 75,725 natural language-SQL pairs, enabling both fine-tuning and rigorous evaluation. Building on this resource, we benchmark reasoning models and language models of varying scales, providing a systematic analysis of their strengths and limitations in financial Text-to-SQL tasks. Finally, we propose a finance-oriented evaluation metric (FINCH Score) that captures nuances overlooked by existing measures, offering a more faithful assessment of model performance.
Complexity vs Empirical Score
- Math Complexity: 2.0/10
- Empirical Rigor: 9.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper is highly data- and implementation-heavy, focusing on dataset curation, model benchmarking, and a new evaluation metric for a practical Text-to-SQL task, with minimal advanced mathematics.
Research Flowchart
flowchart TD A["Research Goal:<br>Address lack of financial Text-to-SQL<br>resources and evaluation"] --> B["Methodology:<br>Curated Financial Dataset Creation"] B --> C["Data Input:<br>FINCH Dataset<br>292 tables / 75,725 NL-SQL pairs"] C --> D["Computational Process:<br>Benchmarking Models<br>&<br>Developing Finance Score"] D --> E["Key Findings:<br>1. Systematic Model Analysis<br>2. FINCH Evaluation Metric<br>3. SOTA Baselines"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper introduces FINCH, a large-scale financial Text-to-SQL dataset, and a finance-oriented evaluation metric called FINCH Score.
Source sentence
we introduce a curated financial dataset (FINCH) comprising 292 tables and 75,725 natural language–SQL pairs, enabling both fine-tuning and rigorous evaluation.
Assumes
No explicit assumptions found in the text.
Data
- FINCH benchmark public
Source sentence
In total, it contains 292 tables, 2233 columns, 177 relations, and 75,725 NL–SQL pairs—providing the largest finance-oriented benchmark, suitable for both evaluation and fine-tuning.
Costs
Not applicable
Finds 3
- GPT-OSS-120B achieves the strongest overall performance, consistently outperforming all other models across most evaluation metrics. performance
Source sentence
Our results reveal that GPT-OSS-120B achieves the strongest overall performance, consistently outperforming all other models across most evaluation metrics.
- Arctic-Text2SQL-R1-7B emerges as the third‑best performer despite its far smaller parameter count, highlighting the value of domain‑specific fine‑tuning. performance
Source sentence
Arctic-Text2SQL-R1-7B emerges as the third-best performer, despite having a far smaller parameter count.
- FINCH Score provides finer‑grained sensitivity to partial correctness compared to strict matching metrics. performance
Source sentence
Comparing Strict Matching (EM+EX) with the FINCH Score illustrates how FINCH introduces finer-grained sensitivity to partial correctness.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 28 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.