Tracing Positional Bias in Financial Decision-Making: Mechanistic Insights from Qwen2.5

ArXiv ID: 2508.18427 “View on arXiv”

Authors: Fabrizio Dimino, Krati Saxena, Bhaskarjit Sarmah, Stefano Pasquali

Abstract

The growing adoption of large language models (LLMs) in finance exposes high-stakes decision-making to subtle, underexamined positional biases. The complexity and opacity of modern model architectures compound this risk. We present the first unified framework and benchmark that not only detects and quantifies positional bias in binary financial decisions but also pinpoints its mechanistic origins within open-source Qwen2.5-instruct models (1.5B-14B). Our empirical analysis covers a novel, finance-authentic dataset revealing that positional bias is pervasive, scale-sensitive, and prone to resurfacing under nuanced prompt designs and investment scenarios, with recency and primacy effects revealing new vulnerabilities in risk-laden contexts. Through transparent mechanistic interpretability, we map how and where bias emerges and propagates within the models to deliver actionable, generalizable insights across prompt types and scales. By bridging domain-specific audit with model interpretability, our work provides a new methodological standard for both rigorous bias diagnosis and practical mitigation, establishing essential guidance for responsible and trustworthy deployment of LLMs in financial systems.

Keywords: Positional bias, Large language models, Mechanistic interpretability, Financial decision making, Model auditing, General Financial Systems

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Street Traders
  • Why: The paper employs sophisticated but applied machine learning concepts (positional bias, mechanistic interpretability) without heavy theoretical derivations or dense mathematical formalisms. It demonstrates high empirical rigor with a novel finance-authentic dataset, cross-scale model testing (1.5B-14B parameters), multiple prompt designs, and a clear path to actionable, implementation-ready insights.
  flowchart TD
    A["Research Goal: Trace positional bias in LLM financial decisions"] --> B["Methodology: Mechanistic interpretability<br/>(Qwen2.5 1.5B-14B)"]
    B --> C["Input: Finance-authentic<br/>binary decision dataset"]
    C --> D["Computational Process: Detect, quantify & trace bias"]
    D --> E["Key Findings:<br/>- Pervasive & scale-sensitive bias<br/>- Mechanistic origins mapped<br/>- Recency/Primacy effects in risk contexts"]