Paper: arXiv 2312.13896
Authors: Hugo Thimonier, Fabrice Popineau, Arpad Rimmel, Bich-Liên Doan, Fabrice Daniel
Abstract
This study explores the application of anomaly detection (AD) methods in imbalanced learning tasks, focusing on fraud detection using real online credit card payment data. We assess the performance of several recent AD methods and compare their effectiveness against standard supervised learning methods. Offering evidence of distribution shift within our dataset, we analyze its impact on the tested models’ performances. Our findings reveal that LightGBM exhibits significantly superior performance across all evaluated metrics but suffers more from distribution shifts than AD methods. Furthermore, our investigation reveals that LightGBM also captures the majority of frauds detected by AD methods. This observation challenges the potential benefits of ensemble methods to combine supervised, and AD approaches to enhance performance. In summary, this research provides practical insights into the utility of these techniques in real-world scenarios, showing LightGBM’s superiority in fraud detection while highlighting challenges related to distribution shifts.
Complexity vs Empirical Score
- Math Complexity: 5.0/10
- Empirical Rigor: 8.5/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper presents a rigorous empirical evaluation on real-world credit card transaction data, analyzing distribution shifts and comparing multiple models with robust metrics. While it introduces advanced concepts in anomaly detection and imbalanced learning, its mathematics remains largely conceptual and algorithmic rather than dense theoretical derivations.
Research Flowchart
flowchart TD
A["Research Goal: Compare AD Methods vs Supervised Learning<br>for Fraud Detection"] --> B["Dataset: Real Online Credit Card Data<br>Imbalanced & Distribution Shift Exists"]
B --> C["Methodology: Evaluate Anomaly Detection<br>vs Standard Supervised Learning"]
C --> D{"Computational Process"}
D --> E["LightGBM Supervised Model"]
D --> F["Anomaly Detection Methods"]
E & F --> G["Key Outcomes"]
G --> H["LightGBM Superior Performance"]
G --> I["AD Methods More Robust<br>to Distribution Shift"]
G --> J["LightGBM Captures<br>Most AD-Detected Frauds"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper empirically evaluates recent anomaly detection methods against LightGBM for fraud detection on real online credit card payment data.
Source sentence
Our findings reveal that LightGBM exhibits significantly superior performance across all evaluated metrics but suffers more from distribution shifts than AD methods.
Assumes 2
- Only fraudulent behaviors change over time, so AD models should be more robust to distribution shift than supervised approaches. model
Source sentence
assuming only fraudulent behaviors change over time, AD models should be more robust to distribution shift than standard supervised approaches.
- If the normal distribution is well characterized, AD models can always exclude new types of anomalies. model
Source sentence
if the normal distribution is well characterized, they should always be able to exclude new types of anomalies.
Data
- Online credit card payments dataset other proprietary
Source sentence
We dispose of a labeled dataset of online credit card payments made available by a major French bank. Our dataset contains 145 features
Costs
Not applicable
Finds 4
- LightGBM consistently outperforms all tested AD methods across various evaluation metrics. performance
Source sentence
Our findings reveal that LightGBM exhibits significantly superior performance across all evaluated metrics but suffers more from distribution shifts than AD methods.
- LightGBM suffers more performance degradation due to distribution shift than AD methods. negative
Source sentence
show that GBDT suffer significantly more from distribution shift than AD methods while displaying substantially better fraud detection performance than all tested AD methods.
- LightGBM captures the majority of frauds detected by AD methods, limiting the benefit of ensembling. performance
Source sentence
LightGBM also captures the majority of frauds detected by AD methods. This observation challenges the potential benefits of ensemble methods to combine supervised, and AD approaches to enhance performance.
- ECOD detected 20.41% of fraud cases not flagged by LightGBM, but 89.02% of ECOD‑detected frauds were also detected by LightGBM. performance
Source sentence
that ECOD did not flag. Notably, the 3.28% represents 10.98% of the fraud cases detected by ECOD. In other words, 89.02% of the fraud cases detected by ECOD were also detected by LightGBM.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 33 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (159)
- Detecting Semantic Anomalies (2019) — link
- Statistical Analysis of Nearest Neighbor Methods for Anomaly Detection (2019) — link
- Beyond Individual Input for Deep Anomaly Detection on Tabular Data (2023) — link
- Trac I n AD : Measuring Influence for Anomaly Detection (2022) — link
- Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion (2010) — link
- Contractive auto-encoders: Explicit invariance during feature extraction (2011)
- Contrastive Multiview Coding (2019) — link
- Momentum Contrast for Unsupervised Visual Representation Learning (2020)
and 151 more
- A Simple Framework for Contrastive Learning of Visual Representations (2020) — link
- PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentation (2021)
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (2018)
- Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection (2019)
- Fixing Bias in Reconstruction-based Anomaly Detection with Lipschitz Discriminators (2022) — link
- Graph Based Anomaly Detection and Description: A Survey (2015) — link
- Informal identification of outliers in medical data (2000)
- A research study on unsupervised machine learning algorithms for early fault detection in predictive maintenance (2018) — link
- A Probabilistic Resource Allocating Network for Novelty Detection (1994) — link
- On Estimation of a Probability Density Function and Mode (1962) — link
- Support Vector Method for Novelty Detection (1999)
- Support Vector Data Description (2004) — link
- Rule Extraction in Unsupervised Anomaly Detection for Model Explainability : Application to OneClass SVM (2020) — link
- TabR: Unlocking the Power of Retrieval-Augmented Tabular Deep Learning (2023) — link
- Classification-Based Anomaly Detection for General Data (2020) — link
- Why do tree-based models still outperform deep learning on typical tabular data? (2022) — link
- Revisiting Deep Learning Models for Tabular Data (2021) — link
- Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning (2021) — link
- XGBoost: A Scalable Tree Boosting System (2016) — link
- Well-tuned Simple Nets Excel on Tabular Datasets (2021)
- Certifiably Adversarially Robust Detection of Out-of-Distribution Data
- Certifiably Adversarially Robust Detection of Out-of-Distribution Data (2021) — link
- Bootstrap your own latent: A new approach to self-supervised Learning (2020) — link
- Exploring Simple Siamese Representation Learning (2020) — link
- Understanding self-supervised Learning Dynamics without Contrastive Pairs (2021) — link
- Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty (2019)
- Deep Learning for Anomaly Detection : A Survey (2019) — link
- The Trace Criterion for Kernel Bandwidth Selection for Support Vector Data Description (2020) — link
- Unsupervised Anomaly Detection with Multi-scale Interpolated Gaussian Descriptors (2021) — link
- Simple and Effective Prevention of Mode Collapse in Deep One-Class Classification (2020) — link
- Anomaly Detection for Tabular Data with Internal Contrastive Learning (2022) — link
- Contrastive Predictive Coding for Anomaly Detection (2021) — link
- Deep Anomaly Detection on Attributed Networks (2019) — link
- Deep Multi-sphere Support Vector Data Description (2020) — link
- Towards Domain-Agnostic Contrastive Learning (2021) — link
- COPOD: Copula-Based Outlier Detection (2020) — link
- Fonctions de repartition an dimensions et leurs marges (1959)
- \ SSD \ : A Unified Framework for Self-Supervised Outlier Detection (2021) — link
- Domain-Agnostic Clustering with Self-Distillation (2021) — link
- DROCC: Deep Robust One-Class Classification (2020) — link
- Statistical Analysis of Nearest Neighbor Methods for Anomaly Detection
- Extended Isolation Forest (2021) — link
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (2018) — link
- Deep Anomaly Detection with Outlier Exposure (2019) — link
- ESAD : End-to-end Deep Semi-supervised Anomaly Detection (2021) — link
- Tabular Data: Deep Learning is Not All You Need (2021) — link
- Adam: A Method for Stochastic Optimization (2015) — link
- Optuna: A Next-generation Hyperparameter Optimization Framework (2019)
- Scikit-learn: Machine Learning in P ython (2011)
- Isolation Forest (2008) — link
- RaPP: Novelty Detection with Reconstruction along Projection Pathway (2020) — link
- Backpropagated Gradient Representations for Anomaly Detection (2020) — link
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (2018) — link
- SpecAE : Spectral AutoEncoder for Anomaly Detection in Attributed Networks (2019) — link
- Isolation Forest (2008) — link
- Open Category Detection with PAC Guarantees (2018) — link
- Explainable Deep One-Class Classification (2021) — link
- Hybrid Models with Deep and Invertible Features (2019) — link
- Deep Weakly-supervised Anomaly Detection (2020) — link
- SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training (2021) — link
- Lightgbm: A highly efficient gradient boosting decision tree (2017)
- The Relationship between Precision-Recall and ROC Curves (2006) — link
- One- Class Classification : A Survey (2021) — link
- Credit Card Fraud Detection in E- Commerce : An Outlier Detection Approach (2019) — link
- FACE : Feasible and Actionable Counterfactual Explanations (2020) — link
- Neural transformation learning for deep anomaly detection beyond images (2021)
- Deep One-Class Classification (2018) — link
- Deep Semi-Supervised Anomaly Detection (2020) — link
- Rethinking Assumptions in Deep Anomaly Detection (2020) — link
- Self- Attentive , Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text (2019) — link
- Self- Attentive , Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text (2019) — link
- A Unifying Review of Deep and Shallow Anomaly Detection (2021) — link
- Detecting Out-of-Distribution Examples with Gram Matrices
- Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery (2017) — link
- Detection of Anomalies in Large Scale Accounting Data Using Deep Autoencoder Networks (2018) — link
- Distributed Top-K Outlier Detection from Astronomy Catalogs using the DEMAC System (2007) — link
- In-Network PCA and Anomaly Detection (2006) — link
- Flow-Based SVDD for Anomaly Detection (2021) — link
- EdgeCentric : Anomaly Detection in Edge-Attributed Networks (2016) — link
- Efficient Anomaly Detection via Matrix Sketching (2018) — link
- Learning and Evaluating Representations for Deep One-class Classification (2021) — link
- Ellipsoidal Subspace Support Vector Data Description (2020) — link
- CSI : Novelty Detection via Contrastive Learning on Distributionally Shifted Instances (2020) — link
- Deep One-Class Classification (2018)
- APATE : A Novel Approach for Automated Credit Card Transaction Fraud Detection Using Network-Based Extensions (2015) — link
- Contrastive Training for Improved Out-of-Distribution Detection (2020) — link
- NetWalk : A Flexible Deep Embedding Approach for Anomaly Detection in Dynamic Networks (2018) — link
- Adversarial Self-Supervised Learning for Out-of-Domain Detection (2021) — link
- Deep Structured Energy Based Models for Anomaly Detection (2016)
- A Practical Guide to Training Restricted Boltzmann Machines (2012) — link
- Hybrid Models for Open Set Recognition (2020) — link
- Deep Residual Flow for Out of Distribution Detection (2020) — link
- DEEP AUTOENCODING GAUSSIAN MIXTURE MODEL FOR UNSUPERVISED ANOMALY DETECTION (2018)
- The Effect of Class Distribution on Classifier Learning: An Empirical Study (2001)
- A Multiple Resampling Method for Learning from Imbalanced Data Sets (2004) — link
- Editorial: Special Issue on Learning from Imbalanced Data Sets (2004) — link
- Experiments with a New Boosting Algorithm (1996)
- A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting (1997) — link
- Cost-Sensitive Boosting for Classification of Imbalanced Data (2007) — link
- Data Mining for Imbalanced Datasets: An Overview (2005) — link
- Issues in Mining Imbalanced Data Sets - A Review Paper (2005)
- Robust random cut forest based anomaly detection on streams (2016) — link
- PIDForest: Anomaly Detection and Certification via Partial Identification (2019) — link
- The Class Imbalance Problem: A Systematic Study (2002)
- Addressing the Curse of Imbalanced Training Sets: One-Sided Selection (2000)
- KNN Approach to Unbalanced Data Distributions: A Case Study Involving Information Extraction (2003)
- A Study of the Behavior of Several Methods for Balancing Machine Learning Training Data (2004) — link
- Applying One-Sided Selection to Unbalanced Datasets. (2000)
- On predicting rare classes with SVM ensembles in scene classification (2003) — link
- Learning and Evaluating Representations for Deep One-Class Classification (2021) — link
- CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances (2020)
- Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (2020) — link
- Mean-Shifted Contrastive Loss for Anomaly Detection (2021) — link
- Towards Domain-Agnostic Contrastive Learning (2021) — link
- Class-boundary alignment for imbalanced dataset learning (2003)
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere (2020)
- Auto-Encoding Variational Bayes (2014) — link
- Nonlinear principal component analysis using autoassociative neural networks (1991)
- Identification of Outliers (1980)
- Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection (2019)
- A robust anomaly finder based on autoencoders (2020) — link
- Neural Networks and Principal Component Analysis: Learning from Examples without Local Minima (1989) — link
- Autoencoders for unsupervised anomaly detection in high energy physics (2021) — link
- Addressing the Curse of Imbalanced Training Sets: One-Sided Selection (1997)
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction (2020) — link
- PyOD: A Python Toolbox for Scalable Outlier Detection (2019) — link
- Evaluating categorical encoding methods on a real credit card fraud detection database (2021) — link
- Visualizing Data using t-SNE (2008) — link
- SMOTE–IPF: Addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering (2015) — link
- Geometric Dataset Distances via Optimal Transport (2020) — link
- Learning from imbalanced data: open challenges and future directions (2016)
- Classification of imbalanced data: a review (2011) — link
- Neural Network Method for failure detection with skewed class distribution (2004) — link
- Applying Support Vector Machines to Imbalanced Data Sets (2004) — link
- Efficient algorithms for mining outliers from large data sets (2000)
- Fast outlier detection in high dimensional spaces (2002)
- Compounds Activity Prediction in Large Imbalanced Datasets with Substructural Relations Fingerprint and EEM (2015)
- Effective detection of sophisticated online banking fraud on extremely imbalanced data (2013) — link
- Generative Adversarial Nets (2014)
- SMOTE: Synthetic Minority Over-sampling Technique (2002) — link
- Learning from Imbalanced Data (2009) — link
- Exploratory Undersampling for Class-Imbalance Learning (2009) — link
- Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning (2005)
- ADASYN: Adaptive synthetic sampling approach for imbalanced learning (2008) — link
- LOF: Identifying Density-Based Local Outliers (2000) — link
- ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions (2022) — link
- ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions (2022) — link
- Extreme Re-Balancing for SVMs: A Case Study (2004) — link
- An Empirical Study on Class Rarity in Big Data (2018) — link
- Extended Isolation Forest (2021) — link
- Learning Long Term Style Preserving Blind Video Temporal Consistency (2021) — link
0 of 159 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.