Page 65 - CT_AI_Class-8

Let us consider a scenario.

A school is developing an AI system to recommend after-school clubs to students. The system uses
past data about which students have enjoyed various clubs. The following training data is used:

Club Boys Girls

Football 60 20
Art 15 55

The AI program studies the data and learns that most boys enjoy playing football and most girls
enjoy art. Now, imagine a new girl student joins the school.
Think: fact bits

 Which club will the AI probably recommend to new students A study by the MIT Media
after studying the given data? Lab found that facial
recognition systems had a
 Is the AI making the recommendation based on ability or 34.7% error rate for dark-
data pattern? skinned women, highlighting

 Do you think every student gets an equal chance through the need for balanced
this AI recommendation system? training data.

 How can the data be improved to help the AI make better
and fairer suggestions?

Ask
AGENT
OrangeAI
If an AI makes a biased decision, who should be held responsible—the AI or the humans who trained it?

Study

ENSURING FAIRNESS

A company once created an AI system to help recruit new employees. The system learned from
past hiring data, which showed that most of the engineers hired in the past were men. Because of
this pattern, the AI started to prefer resumes that looked similar to those of male applicants. This
is a good example of how AI can pick up human biases from historical data if the developers are
not careful.

This situation shows that AI can unintentionally favour certain groups, even though it is supposed
to treat everyone fairly. To prevent this, developers need to ensure that AI systems are trained
with balanced and diverse data.

AI should be designed to benefit everyone, not just a particular group, so that all applicants have
an equal chance, regardless of their gender, race or background. This helps create a fairer and
more inclusive system.

Data and Fairness in AI 63