Leveraging Large Language Models to Democratize Access to Costly Datasets for Academic Research

Paper: arXiv 2412.02065

Authors: Julian Junyan Wang, Victor Xiaoqi Wang

Abstract

Unequal access to costly datasets essential for empirical research has long hindered researchers from disadvantaged institutions, limiting their ability to contribute to their fields and advance their careers. Recent breakthroughs in Large Language Models (LLMs) have the potential to democratize data access by automating data collection from unstructured sources. We develop and evaluate a novel methodology using GPT-4o-mini within a Retrieval-Augmented Generation (RAG) framework to collect data from corporate disclosures. Our approach achieves human-level accuracy in collecting CEO pay ratios from approximately 10,000 proxy statements and Critical Audit Matters (CAMs) from more than 12,000 10-K filings, with LLM processing times of 9 and 40 minutes respectively, each at a cost under US $10. This stands in stark contrast to the hundreds of hours needed for manual collection or the thousands of dollars required for commercial database subscriptions. To foster a more inclusive research community by empowering researchers with limited resources to explore new avenues of inquiry, we share our methodology and the resulting datasets.

Complexity vs Empirical Score

• Math Complexity: 2.0/10
• Empirical Rigor: 8.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: This paper presents a highly practical, empirical approach using LLMs (GPT-4o-mini) and RAG to automate data collection from SEC filings at low cost with extreme accuracy. While mathematical complexity is minimal, the paper excels in empirical rigor, validation against large scale datasets (10,000+ filings), clarity, and exceptional commitment to open science and reproducibility.

Research Flowchart

  flowchart TD
    A[Goal: Democratize access to costly corporate disclosure datasets via LLMs] --> B[Data Inputs: ~10,000 Proxy Statements & ~12,000 10-K Filings]
    B --> C[Methodology: GPT-4o-mini within a Retrieval-Augmented Generation RAG Framework]
    C --> D[Automated Extraction: CEO Pay Ratios & Critical Audit Matters CAMs]
    D --> E[Results: Human-level accuracy achieved in under 40 mins & under $10 per task]
    E --> F[Outcome: Public release of methodology and datasets to empower resource-limited researchers]