Page 161 - Ai_C10_Flipbook
P. 161
• False Positive (Type 1 error): The predicted value was falsely predicted i.e.; the actual value was negative but the
model predicted a positive value.
• False Negative (Type 2 error): The predicted value was falsely predicted i.e.; the actual value was positive but the
model predicted a negative value.
2. Lists the different evaluation model.
Ans. Evaluation techniques involve assessing a machine learning model’s performance on training and test data.
The description of these evaluation models is as follows:
• Overfitting Model: The model (red curve) fits the training data perfectly, including noise, but performs poorly
on the testing data, leading to poor generalisation. In overfitting, the model is too complex and performs well on
training data but poorly on test data. It has low bias and high variance. The model memorizes the training data but
struggles to generalize to new, unseen data.
• Underfitting Model: The model (purple line) is too simplistic, failing to capture the pattern in both the training and
testing data. It has high bias and low variance. The model fails to capture the underlying patterns in the data.
• Perfect Fit Model: The model (green curve) balances complexity and generalisation, fitting the training data well
and performing well on the testing data. It performs well on both training and test data and generalizes effectively
to new data.
• Model Selection: Splitting helps compare models and choose the best one based on performance on the
testing set.
3. What is the purpose of using Precision and Recall together when evaluating a classification model, and how does the
F1 Score help in balancing them?
Ans. Precision and Recall are used together to evaluate how well a model handles both false positives and false negatives.
Precision focuses on how many of the predicted positive cases were actually positive, while Recall measures how many
of the actual positive cases were correctly identified by the model.
The F1 Score is the harmonic mean of Precision and Recall. It helps balance these two metrics by giving a single value
that combines both, making it particularly useful when the costs of false positives and false negatives are important
and need to be minimized equally. F1 Score is often preferred when there's a need to strike a balance between
Precision and Recall rather than focusing on one over the other.
4. What do you understand by accuracy and error in evaluation metrics?
Ans. The term Accuracy is defined as the evaluation metric that measures the total number of predictions that are correct
by the model. It means how close the prediction is to the true value. The accuracy of the model and the performance
of the model is directly proportional, that means better the performance of the model, higher is the accuracy of the
predictions.
The term Error means the action that is inaccurate or wrong. It refers to the difference between a model’s prediction
and the actual outcome. It quantifies how often the model makes mistakes. Based on the error, we choose the machine
learning model that has the best performance for a specific dataset. Low Error in a model performance signifies precise
and reliable predictions.
5. Discuss the ethical concerns around model evaluation.
Ans. The following are the ethical concerns in the model evaluation:
• Bias: Evaluation metrics may fail to detect biases in a model, leading to unfair outcomes. For example, a model might
favor one gender, race, or socio-economic group over another. To prevent this, metrics should be carefully designed
to avoid introducing or perpetuating bias.
• Fairness: Fairness ensures that models treat all groups equally. The evaluation processes must account for fairness
to prevent models from producing discriminatory results.
• Transparency: Sometimes, evaluation models lack clarity, making it difficult to understand how predictions are made.
A transparent approach clearly explains how metrics are chosen and how results are derived.
• Accountability: It is crucial to take responsibility for the choice of evaluation metrics and their outcomes. Providing
clear reasoning behind metric selection helps ensure ethical decision-making and builds trust.
Evaluating Models 159

