Intermediate

AI Bias Audit

Bias auditing evaluates whether an AI system produces unfair outcomes for different demographic groups. It is both a regulatory requirement under laws like NYC Local Law 144 and a fundamental ethical responsibility.

NYC Local Law 144

NYC Local Law 144 (effective July 2023) is the first major U.S. law requiring bias audits for AI systems used in employment decisions:

⚠

Legal Requirements: Employers using Automated Employment Decision Tools (AEDTs) in New York City must: (1) conduct an annual bias audit by an independent auditor, (2) publish the audit results on their website, and (3) notify candidates that an AEDT is being used. Non-compliance carries penalties of $500-$1,500 per violation.

Scope: Applies to tools that substantially assist or replace discretionary decision-making in hiring or promotion
Required metrics: Selection rate and scoring rate for each demographic category (sex, race/ethnicity, and intersectional groups)
Impact ratio: Compare selection/scoring rates between each group and the most favored group. Ratios below 0.8 (the four-fifths rule) indicate potential disparate impact
Independence: The audit must be conducted by an independent auditor with no financial interest in the tool's development

Fairness Metrics

Metric	Definition	When to Use
Demographic Parity	Positive outcomes are equally distributed across groups	When equal representation in outcomes is the goal
Equalized Odds	True positive and false positive rates are equal across groups	When prediction accuracy should not vary by group
Predictive Parity	Positive predictive value is equal across groups	When the meaning of a positive prediction should be consistent
Calibration	Predicted probabilities match actual outcomes for all groups	When risk scores are used for decision-making
Impact Ratio	Selection rate ratio between groups (four-fifths rule)	Employment decisions, regulatory compliance (LL144)

Intersectional Bias Testing

Define Intersectional Groups

Test fairness across combinations of protected attributes (e.g., Black women, Hispanic men, Asian non-binary individuals). Single-axis analysis may mask bias that appears only at intersections.
Handle Small Sample Sizes

Intersectional groups often have small sample sizes. Use appropriate statistical methods: bootstrap confidence intervals, Bayesian approaches, or minimum sample thresholds with disclosure when data is insufficient.
Analyze Patterns

Look for systematic patterns across intersections. If a model disadvantages women and disadvantages racial minorities, the impact on women of color may be compounded and worse than either axis alone.
Report Transparently

Report intersectional results even when they are inconclusive due to sample size. Disclose the limitations of the analysis and recommend data collection strategies to enable future intersectional assessment.

Bias Audit Methodologies

Outcome-Based Testing

Analyze historical decisions to measure outcome rates across demographic groups. Requires access to demographic data and ground truth outcomes. Most commonly used for regulatory compliance.

Counterfactual Testing

Modify protected attributes while keeping other features constant, then measure how predictions change. Reveals whether the model uses protected attributes (directly or via proxies) in its decisions.

Matched Pair Testing

Create pairs of inputs that are identical except for protected attributes. Compare model outputs for each pair. Similar to audit testing used in housing discrimination investigations.

Feature Attribution Analysis

Use SHAP, LIME, or other explainability tools to identify which features drive predictions for different groups. Flag features that may serve as proxies for protected characteristics.

💡

Next Up: In the next lesson, we cover audit documentation - model cards, datasheets, audit reports, and compliance evidence management.