Paper: arXiv 2610.26031

Authors: Jacopo Biggiogera, Nicolas Colombo, Krati Saxena, Bhaskarjit Sarmah, Stefano Pasquali

Abstract

High-stakes tabular pipelines rely on scattered, static quality checks that are hard to evolve and audit against policy intent. Regulated domains particularly suffer as the LLM-era automation raises expectations for traceability and accountability. We propose a vision for an ontology-grounded, repository-native data quality layer in which rules are first-class artifacts: versioned, provenance-linked to governance texts, and re-playable through immutable execution traces. As a first step toward validating this vision, we present a concrete implementation of its agentic rule-discovery core with two rule acquisition modes (Full-Agentic and Guided) and a human-in-the-loop (HITL) review interface. We define an evaluation framework and run experiments addressing whether our fully autonomous, divide-and-conquer agentic rule discovery produces rules of the same quality and diversity as those specified by a human expert, reporting preliminary results on two real world industry datasets in the finance and retail domains.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: This paper proposes a novel agentic approach to data quality rule discovery, addressing a critical need in regulated financial domains. While the mathematical complexity is moderate, the empirical rigor is strong with real-world datasets and a clear evaluation framework. The overall vision and initial implementation show significant promise.

Research Flowchart

  flowchart TD
    A[Research Goal: Auditable Data Quality in Tabular Pipelines] --> B{AuDQ: Agentic Rule Discovery & HITL Review};
    B --> C[Inputs: Ontology-grounded Governance, Finance & Retail Datasets];
    C --> D[Computational Processes: Full-Agentic & Guided Rule Acquisition];
    D --> E[Outcomes: Quality & Diversity of Discovered Rules vs. Human Expert];