Page 239 - AI Ver 3.0 class 10_Flipbook
P. 239

○   Can be alarming, as when AI model is used to predict COVID-19 cases in a residential society, it predicts
                      False Negative i.e. few are infected. Therefore, the society will not be declared as a containment zone thus
                      becoming a reason for sudden rise in COVID-19 cases in that residential society.
                    • Cases where cost of False Positive > False Negative:
                    ○   One  such  case  is  looking  for  a  missing  pet  in  the  nearby  jungle.  Imagine  a  model  telling  you  that  the
                      pet—dog exists in the nearby jungle and you keep on going inside the jungle with all support team but it
                      turns out that it is a false alarm.
                    ○   Model that predicts whether a mail is spam or not. If the model always predicts that the mail is spam, people
                      would not look at it and eventually might lose important information. Missing some important mail might
                      create a problem.
                 Based on different examples we can say, that both Recall and Precision are important. In some cases, you might
                 have a High Precision but Low Recall or Low Precision but High Recall. So, we need an important measure that
                 considers  both  Recall  and  Precision  for the  good  and accurate performance of  an  AI  model.  This  problem  is
                 resolved by considering F1 Score.

                         F1 Score


                 F1 score, also called F-score or F-measure, is a metric used to evaluate the accuracy of a test. It can be defined
                 as the measure of balance between precision and recall. The F1 score ranges between 0 and 1, where 1 indicates
                 perfect Precision and Recall, and 0 indicates a complete failure.
                 A high F1 score means the model has low False Positives (FP) and low False Negatives (FN)—meaning it correctly
                 identifies real cases and minimizes false alarms. It is particularly useful in real-life classification problems, especially
                 when dealing with imbalanced datasets (where one class is much more frequent than the other).

                                                                    Precision × Recall
                                                     F1 Score = 2 ×
                                                                    Precision + Recall

                 A model is said to have a good performance if the F1 Score for that model is high.
                 An ideal situation occurs when both Precision and Recall have value as 1 i.e., 100%, then F1 score would also be an
                 ideal 1 (100%). It is also known as the perfect value for F1 Score.
                 A model is considered to be a total failure when the F1 score is 0.
                 As the values of both Precision and Recall range from 0 to 1, the F1 score also ranges from 0 to 1.

                 The different possibility of the F1 Score is:
                    • If Precision = Low and Recall = Low then F1 Score will be Low.
                    • If Precision = Low and Recall = High then F1 Score will be Low.
                    • If Precision = High and Recall = Low then F1 Score will be Low.
                    • If Precision = High and Recall = High then F1 Score will be High.

                 CASE STUDY: Availability of School Transport


                 In schools, a lot of times it happens that there is no transport facility available to commute. Such unavailability of
                 school transport is a very common and prominent problem. Hence an AI model is designed to predict if there is
                 going to be school transport available or not.







                                                                                           Evaluating Models    237
   234   235   236   237   238   239   240   241   242   243   244