Paper: arXiv 2310.12428
Authors: Joshua Rosaler, Dhruv Desai, Bhaskarjit Sarmah, Dimitrios Vamvourellis, Deran Onay, Dhagash Mehta, Stefano Pasquali
Abstract
We initiate a novel approach to explain the predictions and out of sample performance of random forest (RF) regression and classification models by exploiting the fact that any RF can be mathematically formulated as an adaptive weighted K nearest-neighbors model. Specifically, we employ a recent result that, for both regression and classification tasks, any RF prediction can be rewritten exactly as a weighted sum of the training targets, where the weights are RF proximities between the corresponding pairs of data points. We show that this linearity facilitates a local notion of explainability of RF predictions that generates attributions for any model prediction across observations in the training set, and thereby complements established feature-based methods like SHAP, which generate attributions for a model prediction across input features. We show how this proximity-based approach to explainability can be used in conjunction with SHAP to explain not just the model predictions, but also out-of-sample performance, in the sense that proximities furnish a novel means of assessing when a given model prediction is more or less likely to be correct. We demonstrate this approach in the modeling of US corporate bond prices and returns in both regression and classification cases.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 7.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced statistical concepts like GAP proximities and matrix expansions, requiring dense mathematical notation. It is highly data-driven with a specific case study on corporate bond modeling, but lacks the full backtest implementation details typical of Street Traders.
Research Flowchart
flowchart TD Goal["Research Goal: Explain Random Forest predictions & out-of-sample performance using proximities"] Input["Data Input: US Corporate Bond Prices & Returns"] Method["Methodology: Re-formulate RF as adaptive weighted KNN"] Prox["Compute RF Proximities between data points"] Process1["Local Explainability: Generate attributions across training observations"] Process2["Trust Scoring: Assess likelihood of prediction correctness"] SHAP["Integration: Combine with SHAP (feature-based) for holistic explanation"] Out1["Finding: RF proximities provide novel interpretability layer"] Out2["Finding: Enables assessment of prediction reliability out-of-sample"] Goal --> Input Input --> Method Method --> Prox Prox --> Process1 Prox --> Process2 Process1 --> SHAP Process2 --> SHAP SHAP --> Out1 SHAP --> Out2
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a proximity‑based explainability method for random forests that generates trust scores and complements SHAP by using GAP proximities to assess out‑of‑sample performance.
Source sentence
We initiate a novel approach to explain the predictions and out of sample performance of random forest (RF) regression and classification models by exploiting the fact that any RF can be mathematically formulated as an adaptive weighted K nearest-neighbors model.
Assumes 4
- Random forest predictions can be exactly expressed as a weighted sum of training targets using RF proximities. model
Source sentence
any RF prediction can be rewritten exactly as a weighted sum of the training targets, where the weights are RF proximities between the corresponding pairs of data points.
- The training data is a random sub‑sample of 10,000 trades to enable faster computation of the proximity matrix. data
Source sentence
To enable faster computation of the proximity matrix, the model was trained on a random sub‑sample of 10,000 trades.
- Hyperparameter tuning uses 5‑fold walk‑forward expanding‑window time series cross‑validation. statistical
Source sentence
Hyperparameter tuning was performed using 5-fold walk-forward, expanding-window time series cross validation with randomized grid search over sklearn’s RF regressor max_depth, max_features, min_samples_leaf, and n_estimators parameters.
- Out‑of‑sample performance is evaluated on three months following the nine‑month training period. data
Source sentence
The model is trained on 9 months of public trade data for US investment grade (IG) and high yield (HY) corporate bonds, from Aug. 1 2022 to April 30 2023, and tested on the following three months.
Data
- FINRA TRACE dataset market-prices public
Source sentence
The trade data on which the model was trained was taken from the publicly available FINRA TRACE dataset, while the quotes were drawn from internal fixed income quote data.
- Internal fixed income quote data market-prices proprietary
Source sentence
while the quotes were drawn from internal fixed income quote data.
- Internal bond fundamentals data fundamentals proprietary
Source sentence
Bond characteristics such as sector, industry, ticker, duration, option-adjusted spread, rating, and maturity were also drawn from internal data sources.
Costs
Not applicable
Finds 5
- The RF model improves RMSE by 9.07% over the coarse initial estimate, with test RMSE .356 versus .392 for the baseline. predictive-accuracy
Source sentence
Out of sample, the price estimate generated by the RF model showed a 9.07% improvement in RMSE over the preliminary coarse estimate
- On average only about 1/15 of the training set contributes significantly to each RF prediction. statistical
Source sentence
On average, only 1/15 of the the total training set contributes to the RF model’s prediction.
- Approximately 500 nearest neighbors (out of 7,500 trades) are sufficient to recover the original RF prediction to close precision. statistical
Source sentence
In Figure 1, we see that on average, only about 500 nearest neighbors out of the full training set of 7500 trades are needed to recover the original RF model prediction to close precision.
- Higher proximity‑weighted training MAE correlates with larger out‑of‑sample prediction errors, providing a trust score. performance
Source sentence
The monotonic relationship indicates that higher proximity-weighted training error can help to explain why the model prediction was more likely to be off.
- The proximity‑based explanations complement SHAP by identifying cases where the model is likely to be off due to target variability or data sparsity. theoretical
Source sentence
Proximities identify cases where the model is more likely to be off, either because the target variable shows greater variability and uncertainty in the region around the test point, or because the data is relatively sparse.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 30 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.