Page 155 - Ai_C10_Flipbook
P. 155
Where should we use Precision?
Precision is generally used for unbalanced datasets, when dealing with the False Positives become important, and
the model needs to reduce the FPs as much as possible.
Precision use case examples:
• In email spam detection, False Positives can be costly, leading to missed business opportunities or communication
breakdowns.
• Precision is critical when the cost of a False Positive is high, such as in fraud detection systems where unnecessary
transaction rejections can cause inconvenience to customers.
Task 21 st Century #Critical Thinking
Skills
Now, find out is good Precision equivalent to a good model performance?
Recall
The recall is the measure of our model correctly identifying True Positives. Recall is defined as the fraction of
correctly predicted Positive cases to Total Actual Positive cases. It majorly takes into account the True Actual cases
i.e. it is a measure of our model correctly identifying True Positives.
No. of correct positive predictions TP
Recall = ⇒
Total actual positive values TP+FN
Where should we use Recall?
Recall is generally used for unbalanced dataset, when dealing with the False Negatives become important and the
model needs to reduce the FNs as much as possible.
Recall use case examples:
• Fraud Detection system detects fraudulent transactions in a financial institution. A False Negative would allow
a fraudulent transaction to go undetected, potentially resulting in significant financial loss.
• Fire Alarm system alerts users of potential fires in a building. A False Negative means the system fails to detect
an actual fire, leading to catastrophic consequences.
Which Metric is more Important—Recall or Precision?
• Cases where cost of False Negative > False Positive:
○ Can be riskier and more unfavourable, as nobody expected snowfall and there was no Prediction also but still
snowfall was there.
○ Can be costly and harmful, as when no rain is predicted and the farmers did not start the harvest as they
thought that since no rain is predicted so their wheat crop which is ready to harvest can be done in a day or
two. The model that was supposed to predict the rain but did not do so spoiled the whole crop.
○ Can be alarming, as when AI model is used to predict COVID-19 cases in a residential society, it predicts
False Negative i.e. few are infected. Therefore, the society will not be declared as a containment zone thus
becoming a reason for sudden rise in COVID-19 cases in that residential society.
• Cases where cost of False Positive > False Negative:
○ One such case is looking for a missing pet in the nearby jungle. Imagine a model telling you that the
pet—dog exists in the nearby jungle and you keep on going inside the jungle with all support team but it
turns out that it is a false alarm.
Evaluating Models 153

