Page 161 - Ai_C10_Flipbook
P. 161

•   False Positive (Type 1 error): The predicted value was falsely predicted i.e.; the actual value was negative but the
                         model predicted a positive value.
                       •   False Negative (Type 2 error): The predicted value was falsely predicted i.e.; the actual value was positive but the
                         model predicted a negative value.
                    2.   Lists the different evaluation model.
                   Ans.  Evaluation techniques involve assessing a machine learning model’s performance on training and test data.
                       The description of these evaluation models is as follows:
                       •   Overfitting Model: The model (red curve) fits the training data perfectly, including noise, but performs poorly
                         on the testing data, leading to poor generalisation. In overfitting, the model is too complex and performs well on
                         training data but poorly on test data. It has low bias and high variance. The model memorizes the training data but
                         struggles to generalize to new, unseen data.
                       •   Underfitting Model: The model (purple line) is too simplistic, failing to capture the pattern in both the training and
                         testing data. It has high bias and low variance. The model fails to capture the underlying patterns in the data.
                       •   Perfect Fit Model: The model (green curve) balances complexity and generalisation, fitting the training data well
                         and performing well on the testing data. It performs well on both training and test data and generalizes effectively
                         to new data.
                       •   Model  Selection:  Splitting  helps  compare  models  and  choose  the  best  one  based  on  performance  on  the
                         testing set.

                    3.  What is the purpose of using Precision and Recall together when evaluating a classification model, and how does the
                       F1 Score help in balancing them?

                   Ans.  Precision and Recall are used together to evaluate how well a model handles both false positives and false negatives.
                       Precision focuses on how many of the predicted positive cases were actually positive, while Recall measures how many
                       of the actual positive cases were correctly identified by the model.
                       The F1 Score is the harmonic mean of Precision and Recall. It helps balance these two metrics by giving a single value
                       that combines both, making it particularly useful when the costs of false positives and false negatives are important
                       and  need  to  be  minimized  equally.  F1  Score  is  often  preferred  when  there's  a  need  to  strike  a  balance  between
                       Precision and Recall rather than focusing on one over the other.
                    4.  What do you understand by accuracy and error in evaluation metrics?
                   Ans.  The term Accuracy is defined as the evaluation metric that measures the total number of predictions that are correct
                       by the model. It means how close the prediction is to the true value. The accuracy of the model and the performance
                       of the model is directly proportional, that means better the performance of the model, higher is the accuracy of the
                       predictions.
                        The term Error means the action that is inaccurate or wrong. It refers to the difference between a model’s prediction
                       and the actual outcome. It quantifies how often the model makes mistakes. Based on the error, we choose the machine
                       learning model that has the best performance for a specific dataset. Low Error in a model performance signifies precise
                       and reliable predictions.

                    5.  Discuss the ethical concerns around model evaluation.
                   Ans.  The following are the ethical concerns in the model evaluation:
                        •   Bias: Evaluation metrics may fail to detect biases in a model, leading to unfair outcomes. For example, a model might
                         favor one gender, race, or socio-economic group over another. To prevent this, metrics should be carefully designed
                         to avoid introducing or perpetuating bias.
                        •   Fairness: Fairness ensures that models treat all groups equally. The evaluation processes must account for fairness
                         to prevent models from producing discriminatory results.
                        •   Transparency: Sometimes, evaluation models lack clarity, making it difficult to understand how predictions are made.
                         A transparent approach clearly explains how metrics are chosen and how results are derived.
                        •   Accountability: It is crucial to take responsibility for the choice of evaluation metrics and their outcomes. Providing
                         clear reasoning behind metric selection helps ensure ethical decision-making and builds trust.
                                                                                           Evaluating Models    159
   156   157   158   159   160   161   162   163   164   165   166