Retour

The AI that quietly filtered out women.

Amazon wanted faster hiring.

The AI that quietly filtered

Amazon wanted faster hiring. So they built an AI to scan resumes. It worked. It was fast. And it was biased. It downgraded any CV with “women’s” in it. Why? Because the training data was based on 10 years of mostly male hires. The AI didn’t invent discrimination. It just copied what already existed. Then scaled it.

What went wrong?

Amazon optimized for efficiency. But never stopped to ask: efficient at what? Their AI wasn’t evil. It was obedient. It saw a pattern and followed it, even when that pattern was bias.

How Sensai’s Audit would’ve caught it:

Wrong strategy from the start

They chose speed over fairness.

Fix: We align automation with ethical KPIs, not just ops metrics.

Fixing the wrong layer

They patched resume screening. But upstream data was already flawed.

Fix: We map workflows end-to-end to find the real pressure points.

No bias testing

No fairness constraints. No counterfactuals.

Fix: We inject diversity checks before the first line of code.

Garbage in, garbage out

They trained on biased hiring history.

Fix: We stress-test datasets for representativeness and risk signals.

No team diversity

Tech ran the show. HR and ethics weren’t in the loop.

Fix: We embed multi-disciplinary reviewers into every AI lifecycle.

Poor use-case fit

Hiring is high-risk. AI maturity wasn’t there yet.

Fix: We assess readiness by impact, not by hype.

No recovery plan

When bias was exposed, trust was already gone.

Fix: We build iterative audit loops so issues are caught early, not after scandal.

Why this still matters

If your AI reflects the past, it can’t build the future. And if it’s not tested for bias, it will amplify it. This isn’t just about hiring. It’s about any system making decisions at scale.

Sensai’s Bottom Line

Bad outcomes don’t start with bad people. They start with missing questions. Audit before you automate. Because when AI fails quietly, the damage is loud. Let’s find your blind spots before your customers do.

Your AI is only as good as your audit.

Audit first. Deploy second.

SensAI, Agentic AI dojo.