Paper: arXiv 2308.14634
Authors: Lefteris Loukas, Ilias Stogiannidis, Prodromos Malakasiotis, Stavros Vassos
Abstract
We propose the use of conversational GPT models for easy and quick few-shot text classification in the financial domain using the Banking77 dataset. Our approach involves in-context learning with GPT-3.5 and GPT-4, which minimizes the technical expertise required and eliminates the need for expensive GPU computing while yielding quick and accurate results. Additionally, we fine-tune other pre-trained, masked language models with SetFit, a recent contrastive learning technique, to achieve state-of-the-art results both in full-data and few-shot settings. Our findings show that querying GPT-3.5 and GPT-4 can outperform fine-tuned, non-generative models even with fewer examples. However, subscription fees associated with these solutions may be considered costly for small organizations. Lastly, we find that generative models perform better on the given task when shown representative samples selected by a human expert rather than when shown random ones. We conclude that a) our proposed methods offer a practical solution for few-shot tasks in datasets with limited label availability, and b) our state-of-the-art results can inspire future work in the area.
Complexity vs Empirical Score
- Math Complexity: 3.5/10
- Empirical Rigor: 6.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper relies on standard NLP techniques and prompt engineering with minimal advanced mathematics, but is highly data-driven, using a real-world dataset, comparing model performances, and discussing implementation costs and practical use cases.
Research Flowchart
flowchart TD
A["Research Goal: <br>Few-Shot Text Classification<br>for Financial Domain"] --> B["Data: Banking77 Dataset"]
B --> C{"Modeling Approaches"}
C --> D["In-Context Learning<br>GPT-3.5 & GPT-4"]
C --> E["Fine-Tuning<br>SetFit + Masked LM"]
D --> F{"Key Findings"}
E --> F
F --> G["SOTA Results achieved<br>with SetFit"]
F --> H["Generative Models (GPT)<br>outperform fine-tuned models<br>in few-shot settings"]
F --> I["Human-curated samples<br>superior to random samples"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes using conversational GPT models for easy and quick few-shot text classification in finance and fine‑tuning masked language models with SetFit to achieve state‑of‑the‑art results.
Source sentence
We propose the use of conversational GPT models for easy and quick few-shot text classification in the financial domain using the Banking77 dataset.
Assumes 4
- Limited label availability in financial datasets makes few‑shot methods necessary. data
Source sentence
our proposed methods offer a practical solution for few-shot tasks in datasets with limited label availability
- Subscription fees for GPT APIs may be costly for small organizations. execution
Source sentence
subscription fees associated with these solutions may be considered costly for small organizations
- Token limits restrict the number of examples that can be shown to the model (e.g., 3 samples). model
Source sentence
often have limited token capacity, which only allows us to show the model 3 samples, for example
- Human‑curated representative samples improve in‑context learning performance. model
Source sentence
using our human-curated representative samples leads to better in‑context learning results
Data
- Banking77 text-news public
Source sentence
... a real-life dataset of customer service intents and their classification labels
Costs
Not applicable
Finds 4
- GPT‑3.5 in a 1‑shot setting outperforms S‑MPNet‑v2 and P‑MPNet‑v2 by over 17 percentage points. performance
Source sentence
GPT‑3.5 achieves competitive results despite the fact that it is presented with only 1 sample per class
- GPT‑4 in a 3‑shot setting outperforms all other models by more than 6 percentage points. performance
Source sentence
GPT‑4 also shows potential for few‑shot text classification, outperforming all other models on the 3‑shot setting by more than 6 pp
- In the 10‑shot setting, S‑MPNet‑v2 beats the previous state‑of‑the‑art by 2.2 percentage points. performance
Source sentence
the previous state‑of‑the‑art ... in the 10‑shot setting (by
- Using human‑curated representative samples yields better in‑context learning results than random samples. predictive-accuracy
Source sentence
using our human-curated representative samples leads to better in‑context learning results
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 13 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.