Paper: arXiv 2311.15548
Authors: Haoqiang Kang, Xiao-Yang Liu
Abstract
The hallucination issue is recognized as a fundamental deficiency of large language models (LLMs), especially when applied to fields such as finance, education, and law. Despite the growing concerns, there has been a lack of empirical investigation. In this paper, we provide an empirical examination of LLMs’ hallucination behaviors in financial tasks. First, we empirically investigate LLM model’s ability of explaining financial concepts and terminologies. Second, we assess LLM models’ capacity of querying historical stock prices. Third, to alleviate the hallucination issue, we evaluate the efficacy of four practical methods, including few-shot learning, Decoding by Contrasting Layers (DoLa), the Retrieval Augmentation Generation (RAG) method and the prompt-based tool learning method for a function to generate a query command. Finally, our major finding is that off-the-shelf LLMs experience serious hallucination behaviors in financial tasks. Therefore, there is an urgent need to call for research efforts in mitigating LLMs’ hallucination.
Complexity vs Empirical Score
- Math Complexity: 1.0/10
- Empirical Rigor: 6.0/10 (capped from 8.0: no out-of-sample test; see penalty flags)
- Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 8.0)
Why this score: The paper is primarily empirical, focusing on benchmarking LLM performance and evaluating mitigation techniques with datasets, API calls, and compute metrics, which requires high implementation and data effort. Mathematical complexity is low, as it lacks advanced theoretical derivations and focuses on experimental methodologies rather than dense formulas.
Research Flowchart
flowchart TD Start(["Research Goal:<br>Examine Hallucination in LLMs for Finance"]) --> Input1["Financial Concepts &<br>Terminologies"] Start --> Input2["Historical Stock<br>Price Queries"] Input1 --> Method["Methodology:<br>Empirical Testing of LLM Capabilities"] Input2 --> Method Method --> Mitigate["Mitigation Strategies:<br>1. Few-Shot Learning<br>2. DoLa Decoding<br>3. RAG<br>4. Tool Learning"] Mitigate --> Finding(["Major Finding:<br>Off-the-shelf LLMs show<br>serious hallucination in finance<br>Urgent need for mitigation"]) style Start fill:#e1f5fe,stroke:#01579b,stroke-width:2px style Finding fill:#ffebee,stroke:#c62828,stroke-width:2px
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes evaluating four practical methods—few-shot learning, Decoding by Contrasting Layers (DoLa), Retrieval Augmentation Generation (RAG), and prompt-based tool learning—to mitigate hallucinations in financial tasks.
Source sentence
we evaluate the efficacy of four practical methods, including few-shot learning, Decoding by Contrasting Layers (DoLa), the Retrieval Augmentation Generation (RAG) method and the prompt-based tool learning method
Assumes 1
- The study focuses on factual inconsistency as the primary type of hallucination. other
Source sentence
In this work, we focus particularly on factual inconsistency, as it represents a serious and frequent type of error.
Data
- historical stock price queries market-prices unknown
Source sentence
assess LLM models’ capacity of querying historical stock prices
- acronym recognition and terminology explanation tasks benchmark unknown
Source sentence
Table 1: Results of the acronym recognition task and terminology explanation task
Costs
Not applicable
Finds 5
- Off-the-shelf LLMs experience serious hallucination behaviors in financial tasks. negative
Source sentence
our major finding is that off-the-shelf LLMs experience serious hallucination behaviors in financial tasks
- RAG significantly improves factuality in finance, raising accuracy and FactScore across models. performance
Source sentence
RAG significantly improves the factuality in finance.
- Few-shot prompting reduces errors but still leaves a noticeable gap in MAE and accuracy. performance
Source sentence
while the utilization of few-shot prompting allows models to generate responses that are closer to correct answers, there still exists a gap in terms of MAE and accuracy
- GPT‑3.5‑turbo and GPT‑4 often abstain from answering stock‑symbol and stock‑price questions, avoiding potential misinformation. performance
Source sentence
GPT3.5-turbo and GPT4 opt for a more cautious approach, abstaining from generating responses to some of stock symbol recognition questions and all stock-price-related questions
- Zero‑shot Llama2‑7B and Llama2‑7B‑chat exhibit alarmingly high MAE of 6357.6 USD and 6380.5 USD respectively when predicting stock prices. negative
Source sentence
models such as Llama2-7B and Llama2-7B-chat exhibit alarmingly high Mean Absolute Errors (MAE) of 6357.6 USD and 6380.5 USD, respectively
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.
4 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (50)
- Bloomberggpt: A large language model for finance (2023) — link
- Finbert: A pretrained language model for financial communications (2020) — link
- FinGPT : Democratizing Internet-scale Data for Financial Large Language Models (2023)
- FinGPT : Open-Source Financial Large Language Models — → 2306.06031
- Llama 2: Open foundation and fine-tuned chat models (2023) — link
- Med-halt: Medical domain hallucination test for large language models (2023) — link
- FacTool: Factuality Detection in Generative AI–A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (2023) — link
- Retrieval augmentation reduces hallucination in conversation (2021) — link
and 42 more
- Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection (2023) — link
- Language models (mostly) know what they know (2022) — link
- The internal state of an llm knows when its lying (2023) — link
- Check your facts and try again: Improving large language models with external knowledge and automated feedback (2023) — link
- Rarr: Researching and revising what language models say, using language models (2023)
- Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023) — link
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (2023) — link
- Can Chatgpt Improve Investment Decision? From a Portfolio Management Perspective (2023)
- Toolformer: Language models can teach themselves to use tools (2023) — link
- The curious case of neural text degeneration (2019) — link
- Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models (2023) — link
- Instruct-FinGPT : Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models — → 2306.12659
- Chain-of-Verification Reduces Hallucination in Large Language Models (2023) — link
- Prompting gpt-3 to be reliable (2022) — link
- Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation (2023) — link
- Billion-scale similarity search with gpus (2019)
- Retrieval-augmented generation for knowledge-intensive nlp tasks (2020)
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023) — link
- Llama: Open and efficient foundation language models (2023) — link
- Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023) — link
- PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
- Survey of hallucination in natural language generation (2023)
- A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions (2023) — link
- Trusting Your Evidence: Hallucinate Less with Context-aware Decoding (2023) — link
- Contrastive decoding: Open-ended text generation as optimization (2022) — link
- A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation (2023) — link
- LM vs LM: Detecting Factual Errors via Cross Examination (2023) — link
- Reducing conversational agents’ overconfidence through linguistic calibration (2022)
- Active retrieval augmented generation (2023) — link
- Do Language Models Know When They’re Hallucinating References? (2023) — link
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (2023) — link
- React: Synergizing reasoning and acting in language models (2022) — link
- Gorilla: Large language model connected with massive apis (2023) — link
- Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering (2023) — link
- RARR : Researching and Revising What Language Models Say, Using Language Models (2023) — link
- Teaching language models to support answers with verified quotes (2022) — link
- Performance and exploration of ChatGPT in medical examination, records and education in Chinese: Pave the way for medical AI (2023)
- Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models (2023) — link
- Chatlaw: Open-source legal large language model with integrated external knowledge bases (2023) — link
- Language models are few-shot learners (2020)
- GPT-4 Technical Report (2023)
- Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models — → 2310.04027
3 of 50 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.