Paper: arXiv 2604.02921

Authors: Zhenyu Gao, Wenxi Jiang, Yutong Yan

Abstract

Prior research shows that large language models (LLMs) exhibit systematic extrapolation bias when forming predictions from both experimental and real-world data, and that prompt-based approaches appear limited in alleviating this bias. We propose a supervised fine-tuning (SFT) approach that uses Low-Rank Adaptation (LoRA) to train off-the-shelf LLMs on instruction datasets constructed from rational benchmark forecasts. By intervening at the parameter level, SFT changes how LLMs map observed information into forecasts and thereby mitigates extrapolation bias. We evaluate the fine-tuned model in two settings: controlled forecasting experiments and cross-sectional stock return prediction. In both settings, fine-tuning corrects the extrapolative bias out-of-sample, establishing a low-cost and generalizable method for debiasing LLMs.

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper applies a practical Supervised Fine-Tuning (LoRA) methodology to address behavioral extrapolation bias in LLMs, evaluated across both experimental and actual stock return prediction tasks. While the mathematical formulations are straightforward machine learning techniques rather than complex stochastic calculus, the empirical validation across macro and micro-financial tasks is strong and novel.

Research Flowchart

  flowchart TD
    A[Goal: Mitigate Extrapolation Bias in LLMs] --> B[Input: Instruction Datasets from Rational Forecasts]
    B --> C[Parameter Intervention: Supervised Fine-Tuning via LoRA]
    C --> D[Out-of-Sample Evaluation]
    D --> E[Setting 1: Controlled Forecasting Experiments]
    D --> F[Setting 2: Stock Return Prediction]
    E & F --> G[Outcome: Corrected Extrapolative Bias & Generalizable Debiasing]