The Mechanics of AI Fairness: Comparing Demographic Parity, Equal Opportunity, and Predictive Parity

When an artificial intelligence system denies a mortgage, filters a resume, or flags a medical scan, the immediate human expectation is that the decision was "fair." But fairness is a human philosophy, not a universal mathematical constant. To a machine learning model, equity must be expressed as a rigid equation, and the data shows that these equations are fundamentally at odds with one another.[2]

The AI industry and regulatory bodies have largely coalesced around three primary metrics to measure algorithmic bias: demographic parity, equal opportunity, and predictive parity. Each metric captures a different, intuitively appealing definition of justice. Yet, a foundational mathematical proof in machine learning demonstrates that they cannot all be satisfied at once.[1][3]

The first metric, demographic parity, demands that a model's outcomes are independent of a protected class. If 20% of a loan applicant pool is female, then exactly 20% of the approved loans must go to women. This metric is deeply rooted in civil rights frameworks and is often used as a baseline for auditing systemic bias.[3][4]

However, the evidence shows that demographic parity has severe limitations when the underlying data is already skewed. If one group historically has a lower base rate of qualification—perhaps due to generations of systemic redlining or unequal educational access—enforcing demographic parity requires the model to lower its approval threshold for that specific group. Critics argue this compromises the model's overall accuracy and ignores the ground-truth data.[1][2][5]

Equal opportunity, the second major metric, shifts the focus from equal outcomes to equal accuracy for qualified individuals. It requires that the "true positive rate" is identical across all demographics. Under this definition, a qualified Black applicant and a qualified white applicant must have the exact same mathematical probability of being approved by the algorithm.[1][3]

The limitation of equal opportunity is that it accepts historical imbalances as a given. If fewer individuals in a marginalized group meet the model's definition of "qualified" due to historical inequities, the total number of selected individuals from that group will remain disproportionately low. It ensures fairness at the individual level but does nothing to correct systemic, population-level disparities.[1][2][5]

The third metric, predictive parity, focuses on the model's confidence and reliability. It requires that a positive prediction means exactly the same thing regardless of the applicant's demographic group. If the AI predicts a 70% chance of success for a candidate, that candidate should actually succeed 70% of the time, whether they are male or female.[3][4]

Predictive parity is highly favored in financial and medical sectors where calibration is critical. However, the evidence reveals a critical blind spot: a model can achieve perfect predictive parity while simultaneously rejecting a massive swath of qualified minority candidates, provided the few positive predictions it does make are highly accurate.[1][4]

The central crisis in AI fairness is known as the "impossibility theorem." Mathematical proofs have definitively shown that if the base rates of two groups differ—which is almost always true in real-world sociological data—it is mathematically impossible to satisfy demographic parity, equal opportunity, and predictive parity at the same time.[1][2]

Optimizing for equal opportunity mathematically forces a violation of predictive parity. Conversely, tuning a model to achieve demographic parity will inevitably degrade its equal opportunity score, resulting in qualified individuals from the majority group being rejected to balance the statistical scales.[1][3]

This mathematical reality forces developers and regulators into a zero-sum game. In the criminal justice system, for example, risk assessment tools optimized for predictive parity were found to falsely flag Black defendants at twice the rate of white defendants—a massive failure of equal opportunity that sparked national outrage and highlighted the real-world stakes of these equations.[2][5]

Regulators are currently struggling to map these mathematical trade-offs onto existing legal frameworks. US employment law often leans toward demographic parity, utilizing the "four-fifths rule" to measure disparate impact. Meanwhile, financial regulations often implicitly demand predictive parity to ensure the stability of loan portfolios.[4][5]

The evidence suggests that the choice of fairness metric is ultimately a policy decision masquerading as a math problem. Because the math proves that perfect fairness is impossible, organizations must transparently declare which type of bias they are willing to accept, and which they are legally required to prevent.[1][2][5]