Paper: arXiv 2512.07867

Authors: Masoud Soleimani

Abstract

We develop a transparent and fully auditable LLM-based pipeline for macro-financial stress testing, combining structured prompting with optional retrieval of country fundamentals and news. The system generates machine-readable macroeconomic scenarios for the G7, which cover GDP growth, inflation, and policy rates, and are translated into portfolio losses through a factor-based mapping that enables Value-at-Risk and Expected Shortfall assessment relative to classical econometric baselines. Across models, countries, and retrieval settings, the LLMs produce coherent and country-specific stress narratives, yielding stable tail-risk amplification with limited sensitivity to retrieval choices. Comprehensive plausibility checks, scenario diagnostics, and ANOVA-based variance decomposition show that risk variation is driven primarily by portfolio composition and prompt design rather than by the retrieval mechanism. The pipeline incorporates snapshotting, deterministic modes, and hash-verified artifacts to ensure reproducibility and auditability. Overall, the results demonstrate that LLM-generated macro scenarios, when paired with transparent structure and rigorous validation, can provide a scalable and interpretable complement to traditional stress-testing frameworks.

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper provides a highly rigorous, practical, and well-executed empirical evaluation of a novel LLM-RAG pipeline for financial stress testing. While the mathematical formulations are straightforward (linear factor models, PCA, basic ANOVA), the experimental design, governance layer, diagnostic suite, and statistical validation are exceptionally thorough.

Research Flowchart

  flowchart TD
    A["Research Goal:<br>Auditable LLM-Based Macro-Financial<br>Stress Testing Pipeline"] --> B["Data Inputs:<br>G7 Fundamentals, News Data,<br>Structured Prompts & Portfolios"]
    
    B --> C["Computational Pipeline:<br>Hybrid Prompt-RAG Generation<br>(Snapshotting & Hash Verification)"]
    
    C --> D["Scenario Output:<br>Machine-Readable G7 Metrics<br>(GDP, Inflation, Policy Rates)"]
    
    D --> E["Risk Mapping:<br>Factor-Based Translation to<br>Portfolio VaR & Expected Shortfall"]
    
    E --> F["Validation & Diagnostics:<br>Plausibility Checks, Baselines &<br>ANOVA Variance Decomposition"]
    
    F --> G["Key Outcomes:<br>Coherent Tail-Risk Amplification;<br>Risk Driven by Portfolio & Prompt, Not RAG"]

Cited References (78)

  1. Evaluation of sentiment analysis in finance: from lexicons to transformers (2020)
  2. Natural language based financial forecasting: a survey (2018)
  3. BloombergGPT: A Large Language Model for Finance (2023) — link
  4. Fingpt: Democratizing internet-scale data for financial large language models (2023) — link
  5. The Predictive Edge: Outsmart the Market Using Generative AI and ChatGPT in Financial Forecasting (2024)
  6. Can GPT-4 Sway Experts’ Investment Decisions? (2025)
  7. Risk factor aggregation and stress testing — → 2310.04511
  8. Machine learning interpretability for a stress scenario generation in credit scoring based on counterfactuals (2022)
and 70 more
  1. A machine learning approach in stress testing US bank holding companies (2024)
  2. Retrieval-augmented generation for knowledge-intensive nlp tasks (2020)
  3. LLM Agents for Education: Advances and Applications (2025) — link
  4. Webgpt: Browser-assisted question-answering with human feedback (2021) — link
  5. Atlas: Few-shot learning with retrieval augmented language models (2023)
  6. Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models — → 2310.04027
  7. Stress testing financial systems: macro and micro stress tests, basel standards and value-at-risk as financial stability measures (2022)
  8. Large language models for software engineering: A systematic literature review (2024)
  9. Using an llm to help with code understanding (2024)
  10. Secure coding for web applications: Frameworks, challenges, and the role of LLMs (2025) — link
  11. AI for education (AI4EDU): Advancing personalized education with LLM and adaptive learning (2024)
  12. The end of the policy analyst? Testing the capability of artificial intelligence to generate plausible, persuasive, and useful policy analysis (2024)
  13. Clear: Towards contextual llm-empowered privacy policy analysis and risk generation for large language model applications (2025)
  14. Towards mitigating LLM hallucination via self reflection (2023)
  15. Prospects for inconsistency detection using large language models and sheaves (2024) — link
  16. Measuring and mitigating systemic risks: how the forging of new alliances between central bank and academic economists legitimize the transnational macroprudential agenda (2021)
  17. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers (2020)
  18. The faiss library (2025)
  19. Streamlining Security Vulnerability Triage with Large Language Models (2025) — link
  20. From Insights to Action: Uniting Data and Intellectual Capital for Strategic Success (2025) — link
  21. AI-Generated and YouTube Videos on Navigating the U.S. Healthcare Systems: Evaluation and Reflection (2024)
  22. Stress-Testing Macro Stress Testing: Does It Live Up to Expectations? (2014)
  23. Breaking the Tragedy of the Horizon: Climate Change and Financial Stability (2015)
  24. ``All Scenarios Are Wrong, but Some Are Useful’’—Toward a Framework for Assessing and Using Current Climate Risk Scenarios Within Financial Decisions (2023)
  25. Stress Testing with Multiple Scenarios: A Tale on Tails and Reverse Stress Scenarios (2024)
  26. Real-Time Econometrics (2005)
  27. Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025) — link
  28. A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models (2024) — link
  29. Giving Content to Investor Sentiment: The Role of Media in the Stock Market (2007)
  30. When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks (2011)
  31. FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models (2019) — link
  32. FinBERT: A Pretrained Language Model for Financial Communications (2020) — link
  33. Large Language Models Are Zero-Shot Time Series Forecasters (2023)
  34. Macroeconomic Forecasting with Large Language Models (2024) — link
  35. An MVAR Framework to Capture Extreme Events in Macro-Prudential Stress Tests (2012)
  36. Macro Stress Tests and Crises: What Can We Learn? (2009)
  37. Scenario Generation for Market Risk Models Using Generative Neural Networks (2022)
  38. A Deep Learning Approach for Dynamic Balance Sheet Stress Testing (2022)
  39. Credit Risk Assessment and Financial Decision Support Using Explainable Artificial Intelligence (2024)
  40. Enhancing Credit Risk Prediction Through an Ensemble of Explainable Models (2025)
  41. Self-RAG: Learning to Retrieve, Generate, and Critique Through Self-Reflection (2024)
  42. Corrective Retrieval Augmented Generation (2024)
  43. Principal Component Analysis: A Review and Recent Developments (2016)
  44. Performance Measurement with the Arbitrage Pricing Theory: A New Framework for Analysis (1986)
  45. Common Risk Factors in the Returns on Stocks and Bonds (1993)
  46. World Economic Outlook (WEO) — link
  47. RiskMetrics–Technical Document (1996)
  48. Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation (1982)
  49. Generalized Autoregressive Conditional Heteroskedasticity (1986)
  50. Stress Testing European Banks (2014)
  51. Designing Coherent Scenarios: A Practitioner Perspective (2022)
  52. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing (2021) — link
  53. Modeling the conditional distribution of interest rates as a regime-switching process (1996)
  54. Stock Market Volatility and the Business Cycle (1996)
  55. Macrofinancial stress testing-principles and practices (2012)
  56. Which (Nonlinear) Factor Models? (2023)
  57. The Kernel Trick for Nonlinear Factor Modeling (2022)
  58. Value at risk: the new benchmark for managing financial risk (1997)
  59. Optimization of conditional value-at-risk (2000)
  60. Maximum drawdown (2004)
  61. Monte Carlo methods in financial engineering (2004)
  62. Implementing Value at Risk (2000)
  63. Structured Entity Extraction Using Large Language Models (2024) — link
  64. A reproducibility and generalizability study of large language models for query generation (2024)
  65. Beyond Generalization: A Theory of Robustness in Machine Learning (2023)
  66. Bootstrap methods: another look at the jackknife (1992)
  67. The stationary bootstrap (1994)
  68. Statistical methods for research workers (1970)
  69. Statistical power analysis for the behavioral sciences (2013)
  70. A systematic review of fairness in machine learning (2025)

2 of 78 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.