At some point in almost every model review, someone says a version of: "Can we just make sure it's fair?" It sounds like a reasonable request. It is also, in the precise sense mathematicians and computer scientists have proven, often impossible to fully grant. Not because engineering is sloppy or vendors are hiding something, but because "fair" is not one thing. It is several different, individually reasonable definitions that can point in opposite directions the moment groups have different underlying rates of the outcome you're predicting. This is not a rhetorical flourish — it is a proven mathematical result, and understanding it is the difference between a fairness process that produces defensible decisions and one that produces false comfort.
This piece is for the people who actually own that tradeoff: data scientists building the model, product and risk leads signing off on it, and policy staff who have to explain the decision to a regulator or a board. We'll define the competing metrics in plain language, walk through why you generally can't have all of them, and lay out what teams do instead of chasing an impossible checklist.
What we mean by a "fairness metric"
A fairness metric is a mathematical rule for comparing how a model treats different groups — say, applicants grouped by race, gender, or age. There isn't one such rule; there are several, and they formalize genuinely different intuitions about what "fair" means. Three come up constantly:
Demographic parity (statistical parity). The model should approve (or flag, or select) roughly the same proportion of people from each group, regardless of the outcome it's predicting. If Group A and Group B apply for a loan at the same rate, they should be approved at the same rate.
Equalized odds. Among people who would actually repay the loan, the approval rate should be equal across groups (equal true-positive rates), and among people who would actually default, the rejection rate should be equal across groups (equal false-positive rates). This asks the model to make the same kinds of mistakes at the same rate for everyone, rather than simply approving the same share of people.
Calibration (predictive parity). Among people the model gives the same risk score — say, everyone scored "70% likely to repay" — the actual repayment rate should be about 70%, in every group. A score should mean the same thing regardless of who holds it.
Picture a purely hypothetical lending model deciding who gets approved for a loan, using a risk score from 0-100. Demographic parity asks: are approval rates equal across groups? Equalized odds asks: among people who would have repaid, are approval rates equal — and among people who would have defaulted, are rejection rates equal? Calibration asks: does a score of 70 mean the same real-world repayment probability whether the applicant is in Group A or Group B? Each is a legitimate way to define "fair." None of them is "the" definition, and — this is the crux — a lender usually cannot satisfy all three simultaneously.
The impossibility, in plain English
Here is the part popular coverage of AI bias often skips: when two groups have different base rates for the outcome being predicted — for instance, a genuinely different real-world default rate, or a different underlying prevalence of the condition a model screens for — you generally cannot have a model that is simultaneously calibrated across groups and has equal false-positive and false-negative rates across groups (equalized odds), except in narrow, degenerate cases (such as when the model is a perfect predictor, or the groups happen to have identical base rates).
This was proven independently and near-simultaneously by two research teams. Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan formalized it in "Inherent Trade-Offs in the Fair Determination of Risk Scores" (2016), and Alexandra Chouldechova demonstrated the same conflict empirically in criminal-justice risk scoring in "Fair Prediction with Disparate Impact" (2016-17), where a calibrated recidivism-risk tool produced unequal false-positive rates across racial groups precisely because the underlying base rates differed. The result is a mathematical fact about how probability works, not a bug in a specific vendor's model, and not evidence of bad faith by whoever built it.
Why does this happen? Loosely: a calibrated score has to reflect the real, differing base rates in each group, because that's what "calibrated" means. But equalized odds asks the model's errors to land at the same rate regardless of group. When the base rates genuinely differ, satisfying both constraints at once would require the score to simultaneously track the true group-level rate and ignore it — which is not mathematically consistent. You can pick one. You can sometimes blend toward a compromise. You cannot have all of it, all the time, whenever base rates diverge. And in most real domains — lending, hiring, healthcare triage, criminal justice — base rates do diverge across groups, often for reasons rooted in historical inequity that the model itself didn't create but will reproduce if left unexamined.
It's worth being precise about what this does not mean. It doesn't mean fairness work is pointless, that all models are equally biased, or that regulators are asking for the impossible. It means the question "is this model fair?" is underspecified until you say fair according to which metric, and why that one.
So what do you actually do?
Accepting the impossibility theorem changes the job from "eliminate bias" to "choose, justify, and document a tradeoff." In practice, that looks like four steps.
1. Choose a criterion tied to the real-world harm you're most worried about. If the cost of a false approval (a bad loan, an unqualified hire) and a false denial (a good applicant rejected) fall very differently on different groups, equalized odds — equal error rates — is often the more defensible target. If the harm is that a score itself misleads decision-makers about someone's true risk, calibration matters more. If the goal is remedying historical exclusion from an opportunity altogether, demographic parity may be the right lens, even though it can require treating similarly-scored individuals differently across groups. There is no universally correct answer; the right metric depends on the decision's stakes and who bears the cost of each type of error.
2. Use mitigation techniques, understanding what each trades away. Pre-processing methods reweight or transform training data so a group's representation and outcomes are more balanced before the model ever sees them. In-processing methods, such as adversarial debiasing, build a fairness constraint directly into training. Post-processing methods adjust decision thresholds per group after the model produces a score. Each can improve one metric — but per the impossibility result, doing so will typically move you further from a different metric, and each has its own cost in accuracy or legal exposure that needs to be weighed openly.
3. Move from formal fairness to substantive fairness. A growing body of scholarship argues that chasing statistical metrics in isolation — what Ben Green calls "formal" fairness — can miss the point. In "Escaping the Impossibility of Fairness: From Formal to Substantive Algorithmic Fairness" (Philosophy & Technology, 2022), Green argues that satisfying a metric doesn't guarantee a good outcome for the people affected; you also have to ask whether the model reduces real-world harm and inequity given the social context it operates in, not just whether its numbers balance. More recent work pushes this further: a 2026 FAccT paper, "Revisiting Fairness Impossibility with Endogenous Behavior", shows that even the classic impossibility results can shift once you account for how people change their behavior in response to a model's decisions — a reminder that fairness analysis has to keep evolving alongside how these systems are actually used in the world, not stay frozen at the point of model training. Read alongside a broader framework for what "responsible" deployment looks like, see our explainer on what responsible AI actually requires.
4. Document the choice, the affected groups, and the tradeoff — and make the model's reasoning inspectable. Whatever metric you prioritize, write down why, who reviewed the decision, which groups were analyzed, and what the known tradeoff costs are for the metrics you did not optimize. That record is what turns a defensible technical choice into an accountable one when a regulator, journalist, or affected individual asks how a decision was made. Explainability tooling helps here too — if you can't explain a given decision, you can't meaningfully audit whether the fairness tradeoff you chose is actually holding up in practice; our guide to explainable AI covers the mechanics.
Fairness as an ongoing decision, not a checkbox
The impossibility theorem is not an excuse to give up on fairness — it's the argument for taking it more seriously. A team that understands demographic parity, equalized odds, and calibration can't all be satisfied at once is a team that makes its tradeoff on purpose, with evidence, instead of discovering it by accident after an audit or a lawsuit. That's also the standard the AI Coalition Network applies when we score and rank companies on fairness and accountability practices: not whether a company claims its model is "fair," but whether it can show which definition it chose, why, and what it did about the metrics it didn't optimize. See how organizations compare on our AI accountability rankings, and review the criteria behind those scores in our methodology.