Page 319 - AI Ver 1.0 Class 10
P. 319

✶ Can be unhealthy:  When AI model is used to predict COVID-19 cases in a residential society predicts False
                     Negative i.e. few are infected. Therefore the society will not be declared as a containment zone thus becoming
                     a reason for sudden rise in COVID cases in that residential society.
                    • Cases where cost of  False Positive > False Negative:

                      ✶ One such case is looking for a missing pet in the nearby jungle. Imagine a model telling you that the pet—
                     dog exists in the nearby jungle and you keep on going inside the jungle with all support team but it turns out
                     that it is a false alarm.

                      ✶ Model that predicts whether a mail is spam or not. If the model always predicts that the mail is spam, people
                     would not look at it and eventually might lose important information. Missing some important mails might
                     create a problem.
                 Thus after considering the above examples we conclude that both Recall and Precision are  important. In some
                 cases you might have a High Precision but Low Recall or Low Precision but High Recall. So we need an important
                 measure that considers both Recall and Precision for the good and accurate performance of an AI model. This

                 problem is resolved by considering F1 Score.


                 F1 Score
                 F1 score also called F-score or F-measure is the measure of a test’s accuracy. It can be defined as the measure of
                 balance between precision and recall. The F1 score is a number between 0 and 1 and is the harmonic mean of
                 precision and recall.
                 A good F1 score means that you have low false positives and low false negatives, so you’re correctly identifying
                 real threats, and you are not disturbed by false alarms.
                 F1 score is considered to be a very important and a better evaluation metric in any of the cases of AI model
                 prediction. It is mostly used in real-life classification problems and when imbalanced class distribution
                 exists.
                                                                    Precision × Recall
                                                     F1 Score = 2 ×
                                                                    Precision + Recall

                 A model is said to have a good performance if the F1 Score for that model is high.
                 An ideal situation occurs when both Precision and Recall have value as 1 i.e., 100%, then F1 score would also be an
                 ideal 1 (100%). It is also known as the perfect value for F1 Score. A model is considered to be a total failure when
                 the F1 score is 0.
                 As the values of both Precision and Recall range from 0 to 1, the F1 score also ranges from 0 to 1.

                 The different possibility of the F1 Score is:
                    • If Precision = Low and Recall = Low then F1 Score will be Low.
                    • If Precision = Low and Recall = High then F1 Score will be Low.

                    • If Precision = High and Recall = Low then F1 Score will be Low.
                    • If Precision = High and Recall = High then F1 Score will be High.


                 Case Study
                 In schools, a lot of times it happens that there is no transport facility available to commute. In such unavailability
                 of school transport is a very common and prominent problem. Hence an AI model is designed to predict if there
                 is going to be school transport available or not.



                                                                                                Evaluation  317
   314   315   316   317   318   319   320   321   322   323   324