Learn how to evaluate image classifiers using precision, recall, and probability thresholds for the Microsoft Azure AI-900 exam. Master essential metrics for success.
Table of Contents
Question
You work for Xerigon Corporation which uses the Custom Vision service. You have submitted a large number of images to train your model. You need to evaluate the quality of the classifier.
You have identified the following metrics of the image classifier’s effectiveness:
- Metric 1: The model identified 1,000 images as automobiles, and only 987 of them were automobiles, so the precision is 99.
- Metric 2: In the set of images uploaded to the model, there were 500 images of trucks with a gross vehicle weight rating (GVWR) of 3,500 kg (7,700 lbs). The model identified 410 of them.
- Metric 3: The desired level of confidence that is needed for a prediction to be accepted as correct.
What is the measurement used for each metric?
Match the measurement for each metric that evaluates the model’s image classifier.
Measurement:
- mean average precision (mAP)
- probability threshold
- area under the curve (AUC)
- recall
- precision
Answer
Metric 1: precision
Metric 2: recall
Metric 3: probability threshold
Explanation
There are several metrics that are used to gauge the effectiveness of an image classifier, including precision, recall, mean average precision (mAP), and area under the curve (AUC).
Metric 1 is precision. Precision is the percentage of the objects that the image classifier identified correctly. In this scenario, the model identified 1,000 images as automobiles. However, only 987 of the images were actually automobiles. The precision in this case would be 98.7%.
Metric 2 is recall. Recall is the percentage of classifications that the image classifier got correct. In this scenario, 500 images of trucks with a gross vehicle weight rating (GVWR) of 3500 kg (7,700 lbs) were uploaded to the model. The model identified 410. The recall in this instance is 410/500, which is 82%.
Metric 3 is the probability threshold. The probability threshold is the required level of confidence needed for a prediction to be accepted as correct and is set to 50% by default. When you configure the probability threshold low, more classifications are detected, but there are more false positives. If you configure the probability threshold high, then the precision measurement is high but may hurt the recall measurement. The detected classifications are accurate, but many remain undetected.
You would not choose area under the curve (AUC). This metric uses the area under the Receiver Operating Characteristic (ROC) curve that evaluates the True Positive Rate (TPR) with the False Positive Rate (FPR). AUC values range from 0 to 1. If the AUC is 0.5, then the performance of the model is not higher than random guessing.
You would not choose mean average precision (mAP). This metric uses the area of the precision plotted against the recall that is the average precision (AP). mAP takes the average of AP.
Microsoft Azure AI Engineer Associate AI-102 certification exam practice question and answer (Q&A) dump with detail explanation and reference available free, helpful to pass the Microsoft Azure AI Engineer Associate AI-102 exam and earn Microsoft Azure AI Engineer Associate AI-102 certification.