Page 162 - Ai_C10_Flipbook
P. 162

•   Privacy: Using real-world data for evaluation often involves sensitive personal information, raising concerns about
                       privacy. Measures should be in place to ensure that individual data is protected during the evaluation process.
                     •  Data Protection: Protecting the data used in model evaluation is essential to prevent misuse or unauthorised access.
              C.  Competency-based/Application-based questions.                               21 st  Century   #Critical Thinking
                                                                                                 Skills
                  1.  An AI model made the following predictions for Book Sales forecast. Calculate Accuracy, precision and recall for the
                    following confusion matrix:

                          Confusion Matrix            True Positives            True Negatives

                          Predicted Positive               50                        40

                         Predicted Negative                12                        10


                                     Correction prediction
                Ans.  Accuracy =                         × 100%
                                         Total Cases
                                          (TP + TN)
                             =                           × 100%
                                      (TP + TN + FP + FN)
                                           50 + 10
                             =                           × 100%
                                       50 + 10 + 40 + 12
                                      60
                             =            × 100%
                                     112
                             =      53.5%
                                         True Positive
                    Precision =
                                     All Predicted Positive
                                       TP
                             =
                                     TP + FP
                                       50
                             =
                                     50 + 40

                             =      0.555 or 55.5%
                                           True Positive
                      Recall  =
                                    True Positive + False Negative
                                       50
                             =
                                    50 + 12
                                    50
                             =
                                    62
                             =      0.806 or 80.6%
                  2.  A company is developing a model to predict whether a customer will default on a loan. If the dataset is not split
                    properly into training and testing sets, what issues might arise? How would you ensure fair evaluation?
                Ans. If the dataset is not split properly, the model might overfit, learning patterns from the training data but failing to
                    generalize to new data. This leads to poor performance on real-world predictions. To ensure fair evaluation, the dataset
                    should be split into a training set (for learning patterns) and a testing set (for evaluating generalisation). Sometimes, a
                    validation set is also used for hyperparameter tuning.

                  3.  Identify which metric (Precision or Recall) is to be used in the following cases and why?
                    a.  Email Spam Detection
                    b.  Cancer Diagnosis
                    160     Artificial Intelligence Play (Ver 1.0)-X
   157   158   159   160   161   162   163   164   165   166   167