This is Part 2 of a three-part series on AI governance for UK boards. In Part 1, we explored how AI systems exhibit severe bias – including zero selection rates for Black male candidates – and why “our supplier says it’s fair” isn’t a defence under the Equality Act 2010.
Now boards respond: “But we have human oversight. Doesn’t that prevent discrimination?”
The short answer: No. And the research proves it.
The Comfortable Lie We’ve Been Telling Ourselves
When boards express concern about AI bias, they’re usually given a reassuring answer: “Don’t worry, we have human oversight. A person reviews the AI’s recommendations.”
That sounds sensible. Surely humans can catch what algorithms miss? Surely human judgement is the safety net that prevents discrimination?
January 2025 research from the EU demolished this comforting myth.
The study found humans follow biased AI recommendations just as readily as “fair” AI recommendations. Human oversight alone doesn’t prevent discrimination – it can perpetuate it.
How “Oversight” Becomes Rubber-Stamping
We see this play out in subtle but devastating ways:
An AI system flags CVs for review. The hiring manager gets a shortlist. They review it carefully, exercise their professional judgement, and make selections.
What they don’t see: the hundreds of qualified candidates the algorithm already excluded based on biased pattern-matching. They think they’ve made an independent decision. They haven’t – they’ve rubber-stamped an automated filter they never saw operate.
The EU research involved HR and banking professionals from Italy and Germany making hiring and lending decisions influenced by AI recommendations. The results were stark: even when AI was programmed to be “fair,” it didn’t eliminate the influence of pre-existing human biases in decision-making.
Using “fair” AI reduced gender bias, but human prejudices still shaped final outcomes. And when the AI itself was biased, humans followed along without questioning it.
The Performance Management Trap
Or consider performance management: AI flags “underperformers” based on metrics that systematically disadvantage people with caring responsibilities or disabilities.
Managers review the flags. They have “honest conversations” about performance. They genuinely believe they’re making fair, evidence-based decisions.
But the bias was already baked in. The human conversation just adds a veneer of legitimacy to automated discrimination.
True oversight involves more than just programming AI to be fair or relying on individual judgement.
You need:
Technical measures to ensure AI systems are designed and updated with fairness in mind – not as an afterthought, but as a fundamental requirement with ongoing monitoring.
Organisational strategies that prioritise equity in how AI is deployed, with clear policies on when and how to override algorithmic recommendations.
Governance that establishes guidelines for human-AI collaboration, including protection for people who raise concerns about biased outcomes.
Continuous monitoring of AI-assisted outcomes to identify and address emerging biases before they entrench discrimination.
That’s not most organisations’ current approach. Most treat AI as a “set and forget” technology. They implement it, assume it works fairly because the supplier said so, and only discover the bias when someone sues them.
The Historical Discrimination Problem
Here’s what nobody says in those supplier presentations: AI trained on your organisation’s historical data will learn and scale your past discrimination.
If your organisation has historically promoted fewer women to senior roles, AI will learn that women are less suitable for promotion and recommend accordingly.
If certain demographics are underrepresented in your customer base because of accessibility barriers, AI will learn to target people who look like your existing customers, further excluding others.
If you’ve had racially disparate disciplinary outcomes (even if unintentional), AI will predict that pattern forward and recommend disproportionate discipline for certain groups.
We work with organisations thinking seriously about AI who discover they first need to address the discrimination in their existing practices.
Otherwise they’re just automating unfairness at scale.
That’s uncomfortable. It means acknowledging that your “objective” hiring, promotion, or service allocation processes weren’t objective at all.
But better to face that now than in a tribunal where you’re explaining why your AI systematically disadvantaged protected groups whilst your “human oversight” failed to notice.
What This Means for Your Board
You can’t outsource responsibility for discrimination to an algorithm, then claim human oversight made it acceptable.
Under UK equality law, you’re liable for discriminatory outcomes whether they came from human decisions, algorithmic decisions, or some combination of the two.
The courts have already made clear (in the Mobley v. Workday case discussed in Part 1) that AI’s role in decision-making makes organisations liable for its biases. Adding a human review step doesn’t eliminate that liability if the human is simply ratifying algorithmic discrimination.
What Actually Works
Decision-makers need tools and guidelines to help them understand when and how to override AI recommendations – and organisational support when they do.
That means:
- Training on recognising algorithmic bias (not just general “unconscious bias” training)
- Authority to reject AI recommendations without penalty
- Clear escalation routes when AI produces questionable outcomes
- Regular auditing of who gets screened in vs. screened out
- Monitoring for disparate impact across protected characteristics
- Accountability when oversight fails
Most organisations have none of this. They have AI, they have humans reviewing its outputs, and they assume that’s enough.
It isn’t.
In Part 3 of this series, we’ll give you the comprehensive action plan your board needs to implement immediately – from Monday morning quick wins to long-term capability building.
Read the Complete Series
- Part 1: What Your Board Doesn’t Know About AI Bias
- Part 2: The Myth of Human Oversight (you are here)
- Part 3: Your Board’s AI Action Plan – From Monday Morning to Long-Term Capability
Your Immediate Action
Don’t wait for the full action plan. Do this now:
Audit where AI is being used in your organisation (including tools you might not have classified as “AI”). For each use case, ask: How do we know this isn’t producing discriminatory outcomes? What monitoring exists? Who’s accountable?
If you can’t answer confidently, you have a governance gap that needs immediate attention.
Get in touch to discuss this further
References
- European Commission Joint Research Centre (2025). “Understanding the impact of Human-AI interaction on discrimination.” https://policy-lab.ec.europa.eu/news/understanding-impact-human-ai-interaction-discrimination-2025-01-10_en
- University of Washington (2024). “AI tools show biases in ranking job applicants’ names according to perceived race and gender.” https://www.washington.edu/news/2024/10/31/ai-bias-resume-screening-race-gender/
Jenny Garrett Global
Jenny Garrett OBE is the CEO of Jenny Garrett Global, a highly successful and impactful global leadership, talent and EDI consultancy. She is the author of Equality vs Equity and Rocking Your Role. Her expertise lies in the strategic, human-centric application of leadership in an AI-driven world.
