Page 319 - AI Ver 1.0 Class 10
P. 319
✶ Can be unhealthy: When AI model is used to predict COVID-19 cases in a residential society predicts False
Negative i.e. few are infected. Therefore the society will not be declared as a containment zone thus becoming
a reason for sudden rise in COVID cases in that residential society.
• Cases where cost of False Positive > False Negative:
✶ One such case is looking for a missing pet in the nearby jungle. Imagine a model telling you that the pet—
dog exists in the nearby jungle and you keep on going inside the jungle with all support team but it turns out
that it is a false alarm.
✶ Model that predicts whether a mail is spam or not. If the model always predicts that the mail is spam, people
would not look at it and eventually might lose important information. Missing some important mails might
create a problem.
Thus after considering the above examples we conclude that both Recall and Precision are important. In some
cases you might have a High Precision but Low Recall or Low Precision but High Recall. So we need an important
measure that considers both Recall and Precision for the good and accurate performance of an AI model. This
problem is resolved by considering F1 Score.
F1 Score
F1 score also called F-score or F-measure is the measure of a test’s accuracy. It can be defined as the measure of
balance between precision and recall. The F1 score is a number between 0 and 1 and is the harmonic mean of
precision and recall.
A good F1 score means that you have low false positives and low false negatives, so you’re correctly identifying
real threats, and you are not disturbed by false alarms.
F1 score is considered to be a very important and a better evaluation metric in any of the cases of AI model
prediction. It is mostly used in real-life classification problems and when imbalanced class distribution
exists.
Precision × Recall
F1 Score = 2 ×
Precision + Recall
A model is said to have a good performance if the F1 Score for that model is high.
An ideal situation occurs when both Precision and Recall have value as 1 i.e., 100%, then F1 score would also be an
ideal 1 (100%). It is also known as the perfect value for F1 Score. A model is considered to be a total failure when
the F1 score is 0.
As the values of both Precision and Recall range from 0 to 1, the F1 score also ranges from 0 to 1.
The different possibility of the F1 Score is:
• If Precision = Low and Recall = Low then F1 Score will be Low.
• If Precision = Low and Recall = High then F1 Score will be Low.
• If Precision = High and Recall = Low then F1 Score will be Low.
• If Precision = High and Recall = High then F1 Score will be High.
Case Study
In schools, a lot of times it happens that there is no transport facility available to commute. In such unavailability
of school transport is a very common and prominent problem. Hence an AI model is designed to predict if there
is going to be school transport available or not.
Evaluation 317

