Risk Aware Benchmarking of Large Language Models

Paper: arXiv 2310.07132

Authors: Apoorva Nitsure, Youssef Mroueh, Mattia Rigotti, Kristjan Greenewald, Brian Belgodere, Mikhail Yurochkin, Jiri Navratil, Igor Melnyk, Jerret Ross

Abstract

We propose a distributional framework for benchmarking socio-technical risks of foundation models with quantified statistical significance. Our approach hinges on a new statistical relative testing based on first and second order stochastic dominance of real random variables. We show that the second order statistics in this test are linked to mean-risk models commonly used in econometrics and mathematical finance to balance risk and utility when choosing between alternatives. Using this framework, we formally develop a risk-aware approach for foundation model selection given guardrails quantified by specified metrics. Inspired by portfolio optimization and selection theory in mathematical finance, we define a metrics portfolio for each model as a means to aggregate a collection of metrics, and perform model selection based on the stochastic dominance of these portfolios. The statistical significance of our tests is backed theoretically by an asymptotic analysis via central limit theorems instantiated in practice via a bootstrap variance estimate. We use our framework to compare various large language models regarding risks related to drifting from instructions and outputting toxic content.

Complexity vs Empirical Score

• Math Complexity: 7.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents an innovative cross-disciplinary application of mathematical finance theory—specifically stochastic dominance and portfolio optimization—to LLM risk benchmarking. It maintains high mathematical rigor through asymptotic analysis and bootstrap variance estimation, though empirical validation is applied to AI safety metrics rather than traditional financial asset returns.

Research Flowchart

  flowchart TD
    A[Goal: Risk-Aware LLM Benchmarking with Statistical Significance] --> B[Inputs: LLMs, Instruction Drift & Toxicity Metrics]
    B --> C[Construct Metrics Portfolios per Model]
    C --> D[Evaluate Mean-Risk Trade-offs & Stochastic Dominance]
    D --> E[Compute Statistical Significance via Bootstrap Variance]
    E --> F[Outcomes: Rigorous Model Selection & Guardrail Compliance]