A credit scoring model that discriminates by race. A hiring algorithm that screens out women. A medical diagnosis tool that performs worse on darker-skinned patients. AI bias is not hypothetical — these systems have been deployed and caused real harm. Every data scientist has a professional responsibility to understand bias, detect it, and mitigate it. This guide gives you the practical tools to do that.
Sources of Bias in ML Systems
Bias enters ML systems at multiple stages. Historical bias means the training data reflects past discrimination — if a company historically promoted men over equally qualified women, a model trained on historical promotions will perpetuate this. Representation bias means some groups are underrepresented in training data — a facial recognition system trained mostly on light-skinned faces performs worse on dark-skinned faces. Measurement bias means the features or labels used as proxies are imperfect and systematically wrong for certain groups. Feedback loops mean model predictions influence future data — a model that shows fewer loan offers to a neighbourhood reduces creditworthiness data from that neighbourhood, making the next model even more biased.
Fairness Definitions
There is no single definition of “fair” — different definitions are mathematically incompatible. The three most important: demographic parity (positive prediction rates should be equal across groups), equal opportunity (true positive rates should be equal across groups, so each group benefits equally when they should), and predictive parity (precision should be equal across groups, so the model’s positive predictions are equally accurate). Which definition to use is a business and ethical decision that should involve lawyers, domain experts, and affected communities — not just data scientists.
Detecting Bias with Fairlearn
pip install fairlearn
from fairlearn.metrics import MetricFrame, selection_rate, false_negative_rate
from sklearn.metrics import accuracy_score
# Compute metrics broken down by sensitive attribute
mf = MetricFrame(
metrics={
'accuracy': accuracy_score,
'selection_rate': selection_rate,
'false_negative_rate': false_negative_rate,
},
y_true=y_test,
y_pred=y_pred,
sensitive_features=X_test['gender'] # protected attribute
)
print(mf.by_group)
print(f"
Max disparity in selection rate: {mf.difference()['selection_rate']:.4f}")
Visualising Bias
from fairlearn.metrics import plot_model_comparison
import matplotlib.pyplot as plt
# Fairlearn dashboard
from fairlearn.widget import FairlearnDashboard
FairlearnDashboard(sensitive_features=X_test['gender'],
sensitive_feature_names=['gender'],
y_true=y_test,
y_predicted={'Model': y_pred})
Mitigating Bias
from fairlearn.reductions import ExponentiatedGradient, DemographicParity
# Post-processing: adjust thresholds per group
from fairlearn.postprocessing import ThresholdOptimizer
mitigator = ThresholdOptimizer(
estimator=base_model,
constraints="equalized_odds",
predict_method='predict_proba',
objective='balanced_accuracy_score'
)
mitigator.fit(X_train, y_train, sensitive_features=X_train['gender'])
y_pred_fair = mitigator.predict(X_test, sensitive_features=X_test['gender'])
# In-processing: fairness constraints during training
classifier = ExponentiatedGradient(
LogisticRegression(max_iter=500),
constraints=DemographicParity())
classifier.fit(X_train, y_train, sensitive_features=X_train['gender'])
Explainability with SHAP
import shap
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)
# Global feature importance
shap.summary_plot(shap_values, X_test)
# Local explanation for one prediction
shap.force_plot(explainer.expected_value, shap_values[0], X_test.iloc[0])
# SHAP interaction values (which features interact with gender/race)
shap_interaction = explainer.shap_interaction_values(X_test[:100])
SHAP explanations let you audit model decisions: does the model’s prediction for loan approval rely heavily on zip code (a proxy for race)? SHAP will show this clearly.
Practical Checklist for Ethical ML
Before deploying any model that makes decisions about people, work through this checklist. First, identify all protected attributes in your dataset (race, gender, age, religion, disability status, etc.) and understand which are legally protected in your jurisdiction. Second, compute fairness metrics broken down by protected attribute — not just overall accuracy. Third, document the intended use case and known failure modes in a model card. Fourth, consult with domain experts and, where possible, with people from affected communities. Fifth, set up ongoing monitoring — bias can emerge or worsen as the world changes. Sixth, establish a clear process for users to contest decisions and seek human review.
The Trade-off Between Fairness and Accuracy
Fairness constraints almost always reduce overall accuracy somewhat — there’s a mathematical tension between optimising one metric and satisfying fairness constraints. This trade-off is a business and ethical decision, not a technical one. The question “how much accuracy are we willing to give up for fairness?” should be answered by business leadership with input from legal and ethics teams, not unilaterally by data scientists.
Conclusion
AI ethics is not a soft concern that slows down model development — it’s a hard engineering requirement that prevents real harm and legal liability. Use Fairlearn to detect bias quantitatively, SHAP to audit model decisions for discriminatory patterns, and model cards to document intended use and limitations. The technical tools exist; what’s often missing is the organizational will to use them. Making fairness analysis a standard part of your model development process is the most impactful thing individual data scientists can do.
