Paper: arXiv 2509.08163

Authors: Ho Ming Lee, Katrien Antonio, Benjamin Avanzi, Lorenzo Marchi, Rui Zhou

Abstract

Ensuring equitable treatment (fairness) across protected attributes (such as gender or ethnicity) is a critical issue in machine learning. Most existing literature focuses on binary classification, but achieving fairness in regression tasks-such as insurance pricing or hiring score assessments-is equally important. Moreover, anti-discrimination laws also apply to continuous attributes, such as age, for which many existing methods are not applicable. In practice, multiple protected attributes can exist simultaneously; however, methods targeting fairness across several attributes often overlook so-called “fairness gerrymandering”, thereby ignoring disparities among intersectional subgroups (e.g., African-American women or Hispanic men). In this paper, we propose a distance covariance regularisation framework that mitigates the association between model predictions and protected attributes, in line with the fairness definition of demographic parity, and that captures both linear and nonlinear dependencies. To enhance applicability in the presence of multiple protected attributes, we extend our framework by incorporating two multivariate dependence measures based on distance covariance: the previously proposed joint distance covariance (JdCov) and our novel concatenated distance covariance (CCdCov), which effectively address fairness gerrymandering in both regression and classification tasks involving protected attributes of various types. We discuss and illustrate how to calibrate regularisation strength, including a method based on Jensen-Shannon divergence, which quantifies dissimilarities in prediction distributions across groups. We apply our framework to the COMPAS recidivism dataset and a large motor insurance claims dataset.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper provides a novel distance covariance regularization framework (incorporating JdCov and a novel CCdCov) to tackle intersectional fairness and gerrymandering across multiple continuous and discrete attributes. It combines strong statistical foundations with rigorous empirical applications on insurance claims and recidivism benchmark datasets.

Research Flowchart

  flowchart TD
    A[Goal: Intersectional Fairness in ML<br>Regression & Classification] --> B[Input Data: COMPAS Recidivism<br>& Motor Insurance Claims]
    B --> C[Method: Distance Covariance<br>Regularisation Framework]
    C --> D[Multivariate Dependence Extensions:<br>JdCov & Novel CCdCov]
    D --> E[Calibrate Regularisation Strength<br>via Jensen-Shannon Divergence]
    E --> F[Outcomes: Mitigated Linear/Nonlinear Disparities<br>& Prevented Fairness Gerrymandering]

Cited References (110)

  1. Fairness in Clustering with Multiple Sensitive Attributes (2020) — link
  2. Machine bias risk assessments in criminal sentencing (2016) — link
  3. Guidance Resource: Artificial Intelligence and Discrimination in Insurance Pricing and Underwriting (2022) — link
  4. Understanding Insurance Antidiscrimination Laws (2014) — link
  5. Fairness and Machine Learning: Limitations and Opportunities (2023) — link
  6. Penalizing Unfairness in Binary Classification (2017) — link
  7. A Convex Framework for Fair Regression (2017) — link
  8. A Survey of Bias in Machine Learning through the Prism of Statistical Parity (2022) — link
and 102 more
  1. Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations (2017) — link
  2. Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements (2019) — link
  3. Probability and measure (2017) — link
  4. Pattern recognition and machine learning (2006) — link
  5. Are Algorithms Building the New Infrastructure of Racism? (2021) — link
  6. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification (2018) — link
  7. Building classifiers with independency constraints (2009) — link
  8. Fairness in machine learning: A survey (2024) — link
  9. Improved adversarial learning for fair classification (2019) — link
  10. Data Preprocessing to Mitigate Bias: A Maximum Entropy Based Approach (2020) — link
  11. Distance metrics for measuring joint dependence with application to causal inference (2019) — link
  12. Fairway: A way to build fair ML software (2020) — link
  13. Insurance, biases, discrimination and fairness (2024) — link
  14. Quantifying fairness and discrimination in predictive models (2024) — link
  15. A Fair Classifier Using Kernel Density Estimation (2020) — link
  16. Deep learning with Python (2021) — link
  17. Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification (2019) — link
  18. Optimization and nonsmooth analysis (1990) — link
  19. A computer program used for bail and sentencing decisions was labeled biased against blacks. It’s actually not that clear (2016) — link
  20. Flexibly fair representation learning by disentanglement (2019) — link
  21. Empirical risk minimization under fairness constraints (2018) — link
  22. CASdatasets: Insurance datasets (2022) — link
  23. Council Directive 2004/113/EC of 13 December 2004 implementing the principle of equal treatment between men and women in the access to and supply of goods and services (2004) — link
  24. Artificial Intelligence Act (2024) — link
  25. Algorithmic fairness datasets: the story so far (2022) — link
  26. An intersectional definition of fairness (2020) — link
  27. The discriminating (pricing) actuary (2023) — link
  28. Strictly Proper Scoring Rules, Prediction, and Estimation (2007) — link
  29. Characterizing intersectional group fairness with worst-case comparisons (2021) — link
  30. Non-discriminatory machine learning through convex fairness criteria (2018) — link
  31. A survey on intersectional fairness in machine learning: Notions, mitigation, and challenges (2023) — link
  32. Deep learning (2016) — link
  33. A fair pricing model via adversarial learning (2022) — link
  34. Learning fair representations via distance correlation minimization (2022) — link
  35. Equality of opportunity in supervised learning (2016) — link
  36. Criminal law — sentencing guidelines — Wisconsin Supreme Court requires warning before use of algorithmic risk assessments in sentencing. — State v. Loomis, 881 N.W.2d 749 (Wis. 2016) (2017) — link
  37. Bias mitigation for machine learning classifiers: A comprehensive survey (2024) — link
  38. Bridging fairness gaps: A (conditional) distance covariance perspective in fairness learning (2024) — link
  39. Fast computing for distance covariance (2016) — link
  40. Generalized Demographic Parity for Group Fairness (2022) — link
  41. Applied multivariate statistical analysis (2002)
  42. Fairness issues, current approaches, and challenges in machine learning models (2024) — link
  43. Decision theory for discrimination-aware classification (2012) — link
  44. Fairness-aware learning through regularization approach (2011) — link
  45. InfoFair: Information-theoretic intersectional fairness (2022) — link
  46. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness (2018) — link
  47. Algorithmic bias in recidivism prediction: A causal perspective (2020) — link
  48. Use of ranks in one-criterion variance analysis (1952) — link
  49. On information and sufficiency (1951) — link
  50. Counterfactual fairness (2017) — link
  51. Sample size justification (2022) — link
  52. Deep learning (2015) — link
  53. Divergence measures based on the Shannon entropy (1991) — link
  54. Discrimination-Free Insurance Pricing (2022) — link
  55. What is fair? Proxy discrimination vs. demographic disparities in insurance pricing (2024) — link
  56. Sensitivity-based measures of discrimination in insurance pricing (2024) — link
  57. The balance property in insurance pricing (2024) — link
  58. Practical approaches for fair learning with multitype and multivariate sensitive attributes (2022) — link
  59. A Stochastic Optimization Framework for Fair Risk Minimization (2022) — link
  60. Methods and algorithms for correlation analysis in R (2020) — link
  61. Fairness-aware learning for continuous attributes and treatments (2019) — link
  62. A survey on bias and fairness in machine learning (2021) — link
  63. Bounding and approximating intersectional fairness through marginal fairness (2022) — link
  64. The ranked probability score and the probability score: A comparison (1970) — link
  65. Machine learning: A probabilistic perspective (2012) — link
  66. A Multi-Objective Framework for Balancing Fairness and Accuracy in Debiasing Machine Learning Models (2024) — link
  67. Application of data mining techniques in customer relationship management: A literature review and classification (2009) — link
  68. A Review on Fairness in Machine Learning (2022) — link
  69. Algorithmic Fairness (2023) — link
  70. Post-processing for Individual Fairness (2021) — link
  71. Implementing Anti-Discrimination Policies in Statistical Profiling Models (2011) — link
  72. A Comparative Study of Categorical Variable Encoding Techniques for Neural Network Classifiers (2017) — link
  73. Proxy Discrimination in the Age of Artificial Intelligence and Big Data (2020) — link
  74. Autograd mechanics–PyTorch documentation (2023) — link
  75. A systematic review of fairness in machine learning (2025) — link
  76. Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning (2018) — link
  77. Fast optimization methods for l1 regularization: A comparative study and two new approaches (2007) — link
  78. Understanding machine learning: from theory to algorithms (2014) — link
  79. The chi-square test of distance correlation (2022) — link
  80. Practical Bayesian Optimization of Machine Learning Algorithms (2012) — link
  81. Racial profiling, insurance style: Insurance redlining and the uneven development of metropolitan areas (2003) — link
  82. Fast fair regression via efficient approximations of mutual information (2020) — link
  83. Using effect size—or why the P value is not enough (2012) — link
  84. Brownian distance covariance (2009) — link
  85. Partial distance correlation with methods for dissimilarities (2014) — link
  86. The Energy of Data and Distance Correlation (2023) — link
  87. Measuring and testing dependence by correlation of distances (2007) — link
  88. Algorithmic Accountability Act of 2023 (2023) — link
  89. Achieving Fairness through Adversarial Learning: An Application to Recidivism Prediction (2018) — link
  90. In-Processing Modeling Techniques for Machine Learning Fairness: A Survey (2023) — link
  91. The Discrete Brier and Ranked Probability Skill Scores (2007) — link
  92. Individual Comparisons by Ranking Methods (1992) — link
  93. Bias Regularization in Neural Network Models for General Insurance Pricing (2020) — link
  94. Antidiscrimination Insurance Pricing: Regulations, Fairness Criteria, and Models (2024) — link
  95. To Be Robust or to Be Fair: Towards Fairness in Adversarial Training (2021) — link
  96. Fairness with Overlapping Groups: A Probabilistic Perspective (2020) — link
  97. AdaHessian: An Adaptive Second Order Optimizer for Machine Learning (2021) — link
  98. Fairness Constraints: A Flexible Approach for Fair Classification (2019) — link
  99. Fairness Constraints: Mechanisms for Fair Classification (2017) — link
  100. Learning Fair Representations (2013) — link
  101. Bayes-Optimal Fair Classification with Linear Disparity Constraints via Pre-, In-, and Post-Processing (2024) — link
  102. Towards Fair Classifiers without Sensitive Attributes: Exploring Biases in Related Features (2022) — link

0 of 110 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.