Uncertainty & fairness metrics

Nowadays, many decisions are made taking as suggestion predictive models based on observed data. Even if the learning process is fair and not malicious, predictive models may systematically discriminate against certain groups of people. The resulting decisions will be unfair. Fairness-aware statistical machine learning investigates how to create discrimination-free predictive models. We discuss the various sources of unfairness as solutions to unfairness cannot be separated from its source. We provide a way to distinguish the lack of fairness due to the data generating process/sample from that induced by the predictive algorithm. In addition, we provide some simple strategies to evaluate uncertainty in fairness metrics for the large sample case. An illustrative example on synthetic data from a specific data generating process provides insights into the CART algorithm behaviour.