Paper: arXiv 2504.06293

Authors: Amin Haeri, Jonathan Vitrano, Mahdi Ghelichi

Abstract

Risk management in finance involves recognizing, evaluating, and addressing financial risks to maintain stability and ensure regulatory compliance. Extracting relevant insights from extensive regulatory documents is a complex challenge requiring advanced retrieval and language models. This paper introduces RiskData, a dataset specifically curated for finetuning embedding models in risk management, and RiskEmbed, a finetuned embedding model designed to improve retrieval accuracy in financial question-answering systems. The dataset is derived from 94 regulatory guidelines published by the Office of the Superintendent of Financial Institutions (OSFI) from 1991 to 2024. We finetune a state-of-the-art sentence BERT embedding model to enhance domain-specific retrieval performance typically for Retrieval-Augmented Generation (RAG) systems. Experimental results demonstrate that RiskEmbed significantly outperforms general-purpose and financial embedding models, achieving substantial improvements in ranking metrics. By open-sourcing both the dataset and the model, we provide a valuable resource for financial institutions and researchers aiming to develop more accurate and efficient risk management AI solutions.

Complexity vs Empirical Score

  • Math Complexity: 2.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper provides a practical empirical contribution by curating a domain-specific dataset (RiskData) and fine-tuning a Sentence-BERT embedding model (RiskEmbed) for regulatory RAG applications. It presents minimal theoretical math, focusing instead on empirical evaluation, benchmark comparisons, and open-source accessibility.

Research Flowchart

  flowchart TD
    A[Research Goal: Enhance Financial Risk IR & QA via Domain-Specific RAG] --> B[Data Collection: 94 OSFI Regulatory Guidelines 1991-2024]
    B --> C[Create RiskData Dataset for Fine-Tuning]
    C --> D[Fine-Tune Sentence-BERT Embedding Model]
    D --> E[RiskEmbed Model Development]
    E --> F[Evaluation & Benchmarking against Base/Financial Models]
    F --> G[Outcome: Superior Retrieval Performance & Open-Sourced Resources]

Cited References (47)

  1. Applying BERT to Document Retrieval with Birch (2019) — link
  2. Passage Re-ranking with BERT (2020) — link
  3. ex BERT : Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources (2020) — link
  4. HuggingFace’s Transformers: State-of-the-art Natural Language Processing (2020) — link
  5. NV-Retriever: Improving text embedding models with effective hard-negative mining (2025) — link
  6. Dense Passage Retrieval for Open-Domain Question Answering (2020) — link
  7. Learning Discriminative Projections for Text Similarity Measures (2011) — link
  8. Efficient Natural Language Response Suggestion for Smart Reply (2017) — link
and 39 more
  1. Learning Dense Representations for Entity Retrieval (2019) — link
  2. Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models (2024) — link
  3. BioBERT: a pre-trained biomedical language representation model for biomedical text mining (2019) — link
  4. RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019) — link
  5. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter (2020) — link
  6. Distributed Representations of Words and Phrases and their Compositionality (2013) — link
  7. Deep contextualized word representations (2018) — link
  8. Basel III: A global regulatory framework for more resilient banks and banking systems (2019) — link
  9. Basel III monitoring report (2017) — link
  10. Financial Institutions Management: A Risk Management Approach (2024)
  11. Financial Risk Manager Handbook (2009)
  12. Risk Management and Financial Institutions (2023)
  13. Financial Sentiment Analysis: Techniques and Applications (2024) — link
  14. Text-based sentiment analysis in finance: Synthesising the existing literature and exploring future directions (2024) — link
  15. Predicting stock market using natural language processing (2023) — link
  16. BERT-based Financial Sentiment Index and LSTM-based Stock Return Predictability (2019) — link
  17. MEMORYLLM : Towards Self-Updatable Large Language Models (2024) — link
  18. Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey (2025) — link
  19. Attention is All you Need (2017) — link
  20. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) — link
  21. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019) — link
  22. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding (2019) — link
  23. FACTS About Building Retrieval Augmented Generation-based Chatbots (2024) — link
  24. A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions (2024) — link
  25. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2021) — link
  26. Retrieval-Augmented Generation for Large Language Models: A Survey (2024) — link
  27. A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models (2024) — link
  28. Reducing hallucination in structured outputs via Retrieval-Augmented Generation (2024) — link
  29. Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading (2021) — link
  30. FinQABench (2025) — link
  31. FinanceBench: A New Benchmark for Financial Question Answering (2023) — link
  32. TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance (2021) — link
  33. Finance Alpaca (2023) — link
  34. FIQA Personal Finance Dataset (2024) — link
  35. Stock News Sentiments (2024) — link
  36. ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering (2022) — link
  37. FinQA: A Dataset of Numerical Reasoning over Financial Data (2022) — link
  38. News Stocks (2024) — link
  39. How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection (2023) — link

0 of 47 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.