Paper: arXiv 2311.15548

Authors: Haoqiang Kang, Xiao-Yang Liu

Abstract

The hallucination issue is recognized as a fundamental deficiency of large language models (LLMs), especially when applied to fields such as finance, education, and law. Despite the growing concerns, there has been a lack of empirical investigation. In this paper, we provide an empirical examination of LLMs’ hallucination behaviors in financial tasks. First, we empirically investigate LLM model’s ability of explaining financial concepts and terminologies. Second, we assess LLM models’ capacity of querying historical stock prices. Third, to alleviate the hallucination issue, we evaluate the efficacy of four practical methods, including few-shot learning, Decoding by Contrasting Layers (DoLa), the Retrieval Augmentation Generation (RAG) method and the prompt-based tool learning method for a function to generate a query command. Finally, our major finding is that off-the-shelf LLMs experience serious hallucination behaviors in financial tasks. Therefore, there is an urgent need to call for research efforts in mitigating LLMs’ hallucination.

Complexity vs Empirical Score

  • Math Complexity: 1.0/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: no out-of-sample test; see penalty flags)
  • Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 8.0)

Why this score: The paper is primarily empirical, focusing on benchmarking LLM performance and evaluating mitigation techniques with datasets, API calls, and compute metrics, which requires high implementation and data effort. Mathematical complexity is low, as it lacks advanced theoretical derivations and focuses on experimental methodologies rather than dense formulas.

Research Flowchart

  flowchart TD
  Start(["Research Goal:<br>Examine Hallucination in LLMs for Finance"]) --> Input1["Financial Concepts &<br>Terminologies"]
  Start --> Input2["Historical Stock<br>Price Queries"]
  
  Input1 --> Method["Methodology:<br>Empirical Testing of LLM Capabilities"]
  Input2 --> Method
  
  Method --> Mitigate["Mitigation Strategies:<br>1. Few-Shot Learning<br>2. DoLa Decoding<br>3. RAG<br>4. Tool Learning"]
  
  Mitigate --> Finding(["Major Finding:<br>Off-the-shelf LLMs show<br>serious hallucination in finance<br>Urgent need for mitigation"])
  
  style Start fill:#e1f5fe,stroke:#01579b,stroke-width:2px
  style Finding fill:#ffebee,stroke:#c62828,stroke-width:2px

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes evaluating four practical methods—few-shot learning, Decoding by Contrasting Layers (DoLa), Retrieval Augmentation Generation (RAG), and prompt-based tool learning—to mitigate hallucinations in financial tasks.

Source sentence
we evaluate the efficacy of four practical methods, including few-shot learning, Decoding by Contrasting Layers (DoLa), the Retrieval Augmentation Generation (RAG) method and the prompt-based tool learning method

Assumes 1

  • The study focuses on factual inconsistency as the primary type of hallucination. other
    Source sentence
    In this work, we focus particularly on factual inconsistency, as it represents a serious and frequent type of error.

Data

  • historical stock price queries market-prices unknown
    Source sentence
    assess LLM models’ capacity of querying historical stock prices
  • acronym recognition and terminology explanation tasks benchmark unknown
    Source sentence
    Table 1: Results of the acronym recognition task and terminology explanation task

Costs

Not applicable The paper reports no trading or execution result.

Finds 5

  • Off-the-shelf LLMs experience serious hallucination behaviors in financial tasks. negative
    Source sentence
    our major finding is that off-the-shelf LLMs experience serious hallucination behaviors in financial tasks
  • RAG significantly improves factuality in finance, raising accuracy and FactScore across models. performance
    Source sentence
    RAG significantly improves the factuality in finance.
  • Few-shot prompting reduces errors but still leaves a noticeable gap in MAE and accuracy. performance
    Source sentence
    while the utilization of few-shot prompting allows models to generate responses that are closer to correct answers, there still exists a gap in terms of MAE and accuracy
  • GPT‑3.5‑turbo and GPT‑4 often abstain from answering stock‑symbol and stock‑price questions, avoiding potential misinformation. performance
    Source sentence
    GPT3.5-turbo and GPT4 opt for a more cautious approach, abstaining from generating responses to some of stock symbol recognition questions and all stock-price-related questions
  • Zero‑shot Llama2‑7B and Llama2‑7B‑chat exhibit alarmingly high MAE of 6357.6 USD and 6380.5 USD respectively when predicting stock prices. negative
    Source sentence
    models such as Llama2-7B and Llama2-7B-chat exhibit alarmingly high Mean Absolute Errors (MAE) of 6357.6 USD and 6380.5 USD, respectively

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 4 papers in the archive (4 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (50)

  1. Bloomberggpt: A large language model for finance (2023) — link
  2. Finbert: A pretrained language model for financial communications (2020) — link
  3. FinGPT : Democratizing Internet-scale Data for Financial Large Language Models (2023)
  4. FinGPT : Open-Source Financial Large Language Models — → 2306.06031
  5. Llama 2: Open foundation and fine-tuned chat models (2023) — link
  6. Med-halt: Medical domain hallucination test for large language models (2023) — link
  7. FacTool: Factuality Detection in Generative AI–A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (2023) — link
  8. Retrieval augmentation reduces hallucination in conversation (2021) — link
and 42 more
  1. Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection (2023) — link
  2. Language models (mostly) know what they know (2022) — link
  3. The internal state of an llm knows when its lying (2023) — link
  4. Check your facts and try again: Improving large language models with external knowledge and automated feedback (2023) — link
  5. Rarr: Researching and revising what language models say, using language models (2023)
  6. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023) — link
  7. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (2023) — link
  8. Can Chatgpt Improve Investment Decision? From a Portfolio Management Perspective (2023)
  9. Toolformer: Language models can teach themselves to use tools (2023) — link
  10. The curious case of neural text degeneration (2019) — link
  11. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models (2023) — link
  12. Instruct-FinGPT : Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models — → 2306.12659
  13. Chain-of-Verification Reduces Hallucination in Large Language Models (2023) — link
  14. Prompting gpt-3 to be reliable (2022) — link
  15. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation (2023) — link
  16. Billion-scale similarity search with gpus (2019)
  17. Retrieval-augmented generation for knowledge-intensive nlp tasks (2020)
  18. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023) — link
  19. Llama: Open and efficient foundation language models (2023) — link
  20. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023) — link
  21. PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
  22. Survey of hallucination in natural language generation (2023)
  23. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions (2023) — link
  24. Trusting Your Evidence: Hallucinate Less with Context-aware Decoding (2023) — link
  25. Contrastive decoding: Open-ended text generation as optimization (2022) — link
  26. A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation (2023) — link
  27. LM vs LM: Detecting Factual Errors via Cross Examination (2023) — link
  28. Reducing conversational agents’ overconfidence through linguistic calibration (2022)
  29. Active retrieval augmented generation (2023) — link
  30. Do Language Models Know When They’re Hallucinating References? (2023) — link
  31. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (2023) — link
  32. React: Synergizing reasoning and acting in language models (2022) — link
  33. Gorilla: Large language model connected with massive apis (2023) — link
  34. Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering (2023) — link
  35. RARR : Researching and Revising What Language Models Say, Using Language Models (2023) — link
  36. Teaching language models to support answers with verified quotes (2022) — link
  37. Performance and exploration of ChatGPT in medical examination, records and education in Chinese: Pave the way for medical AI (2023)
  38. Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models (2023) — link
  39. Chatlaw: Open-source legal large language model with integrated external knowledge bases (2023) — link
  40. Language models are few-shot learners (2020)
  41. GPT-4 Technical Report (2023)
  42. Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models — → 2310.04027

3 of 50 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.