Page 157 - Ai_C10_Flipbook
P. 157

Calculate Accuracy, Precision, Recall and F1 Score for the above problem.

                                               (TP+ TN)                                 TP         22       22
                 Classification Accuracy  =                   × 100       Precision =         ⇒          ⇒
                                           (TP+ TN+FP+FN)                             TP +FP     22+12      34
                                                22 18                                     = .647 or 64.7%
                                                   +
                                       =                       × 100

                                           (    22 18 ++  12 +  47)
                                                                                         Precision×Recall        .64 × .31
                                           40                             F1 Score  = 2 ×  Precision+Recall   ⇒ 2 ×   .64 + .31
                                       =       × 100 ⇒ 40.40%
                                           99
                                                                                         0.198
                                             TP       22                                  = 2 ×    ⇒ = 0.41
                                Recall   =        c                                        .95
                                           TP +FN 22 + 47
                                           22
                                       =      ⇒ 0.318 or 31.8%
                                           69
                         Ethical Concerns Around Model Evaluation


                 The following are the ethical concerns in the model evaluation:
                    • Bias - Evaluation metrics may fail to detect biases in a model, leading to unfair outcomes. For example, a
                   model might favor one gender, race, or socio-economic group over another. To prevent this, metrics should be
                   carefully designed to avoid introducing or perpetuating bias.
                    • Fairness - Fairness ensures that models treat all groups equally. The evaluation processes must account for
                   fairness to prevent models from producing discriminatory results.
                    • Transparency - Sometimes, evaluation models lack clarity, making it difficult to understand how predictions are
                   made. A transparent approach clearly explains how metrics are chosen and how results are derived.
                    • Accountability - It is crucial to take responsibility for the choice of evaluation metrics and their outcomes.
                   Providing clear reasoning behind metric selection helps ensure ethical decision-making and builds trust.
                    • Privacy - Using real-world data for evaluation often involves sensitive personal information, raising concerns about
                   privacy. Measures should be in place to ensure that individual data is protected during the evaluation process.
                    • Data Protection - Protecting the data used in model evaluation is essential to prevent misuse or unauthorised
                   access.

                           At a Glance



                       • Model evaluation is the process of applying various metrics to assess a machine learning model’s performance.
                       • Training subset is used to make the model learn patterns from the data. It comprises 70% to 80% of the dataset.
                       • Testing subset is used to evaluate a model on the unseen data. It comprises of 20% to 30% of the dataset.
                       • The ML algorithm is trained using the training data. This involves feeding the data into the algorithm, which learns
                       patterns and relationships to create a model.
                       • Evaluation techniques involve assessing a machine learning model’s performance on training and test data.
                       • The  accuracy  of  the  model  and  the  performance  of  the  model  is  directly  proportional,  that  means  better  the
                       performance of the model, higher is the accuracy of the predictions.
                       • The term Error means the action that is inaccurate or wrong. It refers to the difference between a model’s prediction
                       and the actual outcome.
                       • Classification is a type of supervised learning in machine learning where the goal is to predict the categorical label
                       or class of a given input based on historical data.
                       • Classification is part of supervised machine learning in which we put labelled data for training.
                       • Some popular metrics used for classification models are confusion matrix, classification accuracy, precision and recall.
                       • The classification outcomes based on the different values of actual and predicted labels are True Positive, True
                       Negative, False Positive and False Negative.
                       • Precision is the ratio of True Positive cases to All predicted positive cases.
                       • F1 score can be defined as the measure of balance between precision and recall.


                                                                                           Evaluating Models    155
   152   153   154   155   156   157   158   159   160   161   162