Paper: arXiv 2310.12428

Authors: Joshua Rosaler, Dhruv Desai, Bhaskarjit Sarmah, Dimitrios Vamvourellis, Deran Onay, Dhagash Mehta, Stefano Pasquali

Abstract

We initiate a novel approach to explain the predictions and out of sample performance of random forest (RF) regression and classification models by exploiting the fact that any RF can be mathematically formulated as an adaptive weighted K nearest-neighbors model. Specifically, we employ a recent result that, for both regression and classification tasks, any RF prediction can be rewritten exactly as a weighted sum of the training targets, where the weights are RF proximities between the corresponding pairs of data points. We show that this linearity facilitates a local notion of explainability of RF predictions that generates attributions for any model prediction across observations in the training set, and thereby complements established feature-based methods like SHAP, which generate attributions for a model prediction across input features. We show how this proximity-based approach to explainability can be used in conjunction with SHAP to explain not just the model predictions, but also out-of-sample performance, in the sense that proximities furnish a novel means of assessing when a given model prediction is more or less likely to be correct. We demonstrate this approach in the modeling of US corporate bond prices and returns in both regression and classification cases.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced statistical concepts like GAP proximities and matrix expansions, requiring dense mathematical notation. It is highly data-driven with a specific case study on corporate bond modeling, but lacks the full backtest implementation details typical of Street Traders.

Research Flowchart

  flowchart TD
  Goal["Research Goal: Explain Random Forest predictions & out-of-sample performance using proximities"]
  
  Input["Data Input: US Corporate Bond Prices & Returns"]
  
  Method["Methodology: Re-formulate RF as adaptive weighted KNN"]
  Prox["Compute RF Proximities between data points"]
  
  Process1["Local Explainability: Generate attributions across training observations"]
  Process2["Trust Scoring: Assess likelihood of prediction correctness"]
  
  SHAP["Integration: Combine with SHAP (feature-based) for holistic explanation"]
  
  Out1["Finding: RF proximities provide novel interpretability layer"]
  Out2["Finding: Enables assessment of prediction reliability out-of-sample"]
  
  Goal --> Input
  Input --> Method
  Method --> Prox
  Prox --> Process1
  Prox --> Process2
  Process1 --> SHAP
  Process2 --> SHAP
  SHAP --> Out1
  SHAP --> Out2

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a proximity‑based explainability method for random forests that generates trust scores and complements SHAP by using GAP proximities to assess out‑of‑sample performance.

Source sentence
We initiate a novel approach to explain the predictions and out of sample performance of random forest (RF) regression and classification models by exploiting the fact that any RF can be mathematically formulated as an adaptive weighted K nearest-neighbors model.

Assumes 4

  • Random forest predictions can be exactly expressed as a weighted sum of training targets using RF proximities. model
    Source sentence
    any RF prediction can be rewritten exactly as a weighted sum of the training targets, where the weights are RF proximities between the corresponding pairs of data points.
  • The training data is a random sub‑sample of 10,000 trades to enable faster computation of the proximity matrix. data
    Source sentence
    To enable faster computation of the proximity matrix, the model was trained on a random sub‑sample of 10,000 trades.
  • Hyperparameter tuning uses 5‑fold walk‑forward expanding‑window time series cross‑validation. statistical
    Source sentence
    Hyperparameter tuning was performed using 5-fold walk-forward, expanding-window time series cross validation with randomized grid search over sklearn’s RF regressor max_depth, max_features, min_samples_leaf, and n_estimators parameters.
  • Out‑of‑sample performance is evaluated on three months following the nine‑month training period. data
    Source sentence
    The model is trained on 9 months of public trade data for US investment grade (IG) and high yield (HY) corporate bonds, from Aug. 1 2022 to April 30 2023, and tested on the following three months.

Data

  • FINRA TRACE dataset · FINRA market-prices public
    Aug. 1 2022 to April 30 2023 · US investment grade and high yield corporate bonds
    Source sentence
    The trade data on which the model was trained was taken from the publicly available FINRA TRACE dataset, while the quotes were drawn from internal fixed income quote data.
  • Internal fixed income quote data market-prices proprietary
    Source sentence
    while the quotes were drawn from internal fixed income quote data.
  • Internal bond fundamentals data fundamentals proprietary
    Source sentence
    Bond characteristics such as sector, industry, ticker, duration, option-adjusted spread, rating, and maturity were also drawn from internal data sources.

Costs

Not applicable The paper reports no trading or execution result.

Finds 5

  • The RF model improves RMSE by 9.07% over the coarse initial estimate, with test RMSE .356 versus .392 for the baseline. predictive-accuracy
    Source sentence
    Out of sample, the price estimate generated by the RF model showed a 9.07% improvement in RMSE over the preliminary coarse estimate
  • On average only about 1/15 of the training set contributes significantly to each RF prediction. statistical
    Source sentence
    On average, only 1/15 of the the total training set contributes to the RF model’s prediction.
  • Approximately 500 nearest neighbors (out of 7,500 trades) are sufficient to recover the original RF prediction to close precision. statistical
    Source sentence
    In Figure 1, we see that on average, only about 500 nearest neighbors out of the full training set of 7500 trades are needed to recover the original RF model prediction to close precision.
  • Higher proximity‑weighted training MAE correlates with larger out‑of‑sample prediction errors, providing a trust score. performance
    Source sentence
    The monotonic relationship indicates that higher proximity-weighted training error can help to explain why the model prediction was more likely to be off.
  • The proximity‑based explanations complement SHAP by identifying cases where the model is likely to be off due to target variability or data sparsity. theoretical
    Source sentence
    Proximities identify cases where the model is more likely to be off, either because the target variable shows greater variability and uncertainty in the region around the test point, or because the data is relatively sparse.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 30 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.