Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance

ArXiv ID: 2308.14634 “View on arXiv”

Authors: Unknown

Abstract

We propose the use of conversational GPT models for easy and quick few-shot text classification in the financial domain using the Banking77 dataset. Our approach involves in-context learning with GPT-3.5 and GPT-4, which minimizes the technical expertise required and eliminates the need for expensive GPU computing while yielding quick and accurate results. Additionally, we fine-tune other pre-trained, masked language models with SetFit, a recent contrastive learning technique, to achieve state-of-the-art results both in full-data and few-shot settings. Our findings show that querying GPT-3.5 and GPT-4 can outperform fine-tuned, non-generative models even with fewer examples. However, subscription fees associated with these solutions may be considered costly for small organizations. Lastly, we find that generative models perform better on the given task when shown representative samples selected by a human expert rather than when shown random ones. We conclude that a) our proposed methods offer a practical solution for few-shot tasks in datasets with limited label availability, and b) our state-of-the-art results can inspire future work in the area.

Keywords: few-shot text classification, GPT-3.5, GPT-4, in-context learning, SetFit, General / NLP

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 6.5/10
  • Quadrant: Street Traders
  • Why: The paper relies on standard NLP techniques and prompt engineering with minimal advanced mathematics, but is highly data-driven, using a real-world dataset, comparing model performances, and discussing implementation costs and practical use cases.
  flowchart TD
    A["Research Goal: <br>Few-Shot Text Classification<br>for Financial Domain"] --> B["Data: Banking77 Dataset"]
    B --> C{"Modeling Approaches"}
    C --> D["In-Context Learning<br>GPT-3.5 & GPT-4"]
    C --> E["Fine-Tuning<br>SetFit + Masked LM"]
    D --> F{"Key Findings"}
    E --> F
    F --> G["SOTA Results achieved<br>with SetFit"]
    F --> H["Generative Models (GPT)<br>outperform fine-tuned models<br>in few-shot settings"]
    F --> I["Human-curated samples<br>superior to random samples"]