Behavioral Consistency Validation for LLM Agents: An Analysis of Trading-Style Switching through Stock-Market Simulation

Paper: arXiv 2602.07023

Authors: Zeping Li, Guancheng Wan, Keyang Chen, Yu Chen, Yiwen Zhao, Philip Torr, Guangnan Ye, Zhenfei Yin, Hongfeng Chai

Abstract

Recent works have increasingly applied Large Language Models (LLMs) as agents in financial stock market simulations to test if micro-level behaviors aggregate into macro-level phenomena. However, a crucial question arises: Do LLM agents’ behaviors align with real market participants? This alignment is key to the validity of simulation results. To explore this, we select a financial stock market scenario to test behavioral consistency. Investors are typically classified as fundamental or technical traders, but most simulations fix strategies at initialization, failing to reflect real-world trading dynamics. In this work, we assess whether agents’ strategy switching aligns with financial theory, providing a framework for this evaluation. We operationalize four behavioral-finance drivers-loss aversion, herding, wealth differentiation, and price misalignment-as personality traits set via prompting and stored long-term. In year-long simulations, agents process daily price-volume data, trade under a designated style, and reassess their strategy every 10 trading days. We introduce four alignment metrics and use Mann-Whitney U tests to compare agents’ style-switching behavior with financial theory. Our results show that recent LLMs’ switching behavior is only partially consistent with behavioral-finance theories, highlighting the need for further refinement in aligning agent behavior with financial theory.

Complexity vs Empirical Score

• Math Complexity: 2.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper introduces a novel framework for evaluating whether LLM agent style-switching behaviors align with behavioral finance theories using market simulation and hypothesis testing. While mathematically light, it offers solid empirical evaluation through factorial agent initialization and non-parametric statistical tests (Mann-Whitney U).

Research Flowchart

  flowchart TD
    A[Research Goal: Validate LLM Agent<br/>Trading-Style Switching Consistency] --> B[Inputs: Daily Price-Volume Data &<br/>4 Behavioral Finance Traits via Prompts]
    B --> C[Simulation: 1-Year Stock Market<br/>Agents reassess strategy every 10 days]
    C --> D[Evaluation: Calculate 4 Alignment Metrics &<br/>Perform Mann-Whitney U Tests]
    D --> E[Key Outcome: LLM switching is only<br/>PARTIALLY consistent with financial theory]

Cited References (42)

1. Dynamic models of segregation (1971)
2. Growing artificial societies: social science from the bottom up (1996)
3. Agent-based modeling: Methods and techniques for simulating human systems (2002)
4. Llms and generative agent-based models for complex systems research (2024)
5. A survey of large language models (2023) — link
6. Chain-of-thought prompting elicits reasoning in large language models (2022)
7. Character-llm: A trainable agent for role-playing (2023) — link
8. Camel: Communicative agents for" mind" exploration of large language model society (2023)

3 of 42 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.