Page 157 - Ai_C10_Flipbook
P. 157
Calculate Accuracy, Precision, Recall and F1 Score for the above problem.
(TP+ TN) TP 22 22
Classification Accuracy = × 100 Precision = ⇒ ⇒
(TP+ TN+FP+FN) TP +FP 22+12 34
22 18 = .647 or 64.7%
+
= × 100
( 22 18 ++ 12 + 47)
Precision×Recall .64 × .31
40 F1 Score = 2 × Precision+Recall ⇒ 2 × .64 + .31
= × 100 ⇒ 40.40%
99
0.198
TP 22 = 2 × ⇒ = 0.41
Recall = c .95
TP +FN 22 + 47
22
= ⇒ 0.318 or 31.8%
69
Ethical Concerns Around Model Evaluation
The following are the ethical concerns in the model evaluation:
• Bias - Evaluation metrics may fail to detect biases in a model, leading to unfair outcomes. For example, a
model might favor one gender, race, or socio-economic group over another. To prevent this, metrics should be
carefully designed to avoid introducing or perpetuating bias.
• Fairness - Fairness ensures that models treat all groups equally. The evaluation processes must account for
fairness to prevent models from producing discriminatory results.
• Transparency - Sometimes, evaluation models lack clarity, making it difficult to understand how predictions are
made. A transparent approach clearly explains how metrics are chosen and how results are derived.
• Accountability - It is crucial to take responsibility for the choice of evaluation metrics and their outcomes.
Providing clear reasoning behind metric selection helps ensure ethical decision-making and builds trust.
• Privacy - Using real-world data for evaluation often involves sensitive personal information, raising concerns about
privacy. Measures should be in place to ensure that individual data is protected during the evaluation process.
• Data Protection - Protecting the data used in model evaluation is essential to prevent misuse or unauthorised
access.
At a Glance
• Model evaluation is the process of applying various metrics to assess a machine learning model’s performance.
• Training subset is used to make the model learn patterns from the data. It comprises 70% to 80% of the dataset.
• Testing subset is used to evaluate a model on the unseen data. It comprises of 20% to 30% of the dataset.
• The ML algorithm is trained using the training data. This involves feeding the data into the algorithm, which learns
patterns and relationships to create a model.
• Evaluation techniques involve assessing a machine learning model’s performance on training and test data.
• The accuracy of the model and the performance of the model is directly proportional, that means better the
performance of the model, higher is the accuracy of the predictions.
• The term Error means the action that is inaccurate or wrong. It refers to the difference between a model’s prediction
and the actual outcome.
• Classification is a type of supervised learning in machine learning where the goal is to predict the categorical label
or class of a given input based on historical data.
• Classification is part of supervised machine learning in which we put labelled data for training.
• Some popular metrics used for classification models are confusion matrix, classification accuracy, precision and recall.
• The classification outcomes based on the different values of actual and predicted labels are True Positive, True
Negative, False Positive and False Negative.
• Precision is the ratio of True Positive cases to All predicted positive cases.
• F1 score can be defined as the measure of balance between precision and recall.
Evaluating Models 155

