Using Annotation Consolidation Functions for Accurate Labelling Tasks
Question
You work as a machine learning specialist for a media sharing service.
Healthcare professionals will use the media sharing service to share images of x-rays, MRIs, and other medical imagery.
The accuracy of labelling these images is of primary importance, since the labelling will be used in auto diagnostic software.
As your team builds the data repository to be used by your machine learning algorithms, you need to use human manual labellers.
You have decided to use Amazon Ground Truth for this purpose.
Since accuracy is of prime importance, you have decided to use the annotation consolidation feature of Ground Truth to ensure proper labelling of the medical images. Which of the Ground Truth annotation consolidation functions should you use to ensure the accuracy of your labelling tasks? (Select TWO)
Answers
Explanations
Click on the arrows to vote for the correct answer
A. B. C. D. E.Answers: A, B.
Option A is correct.
The bounding box finds the most similar bounding boxes from workers and averages them, thus using the power of multiple workers to annotate your images more accurately.
Option B is correct.
The semantic segmentation feature fuses the pixel annotations of multiple workers and applying a smoothing function to the image, thus using the power of multiple workers to annotate your images more accurately.
Option C is incorrect.
The named entity feature is used with text annotation work, not image annotation.
Option D is incorrect.
The Ground Truth output manifest allows the output of a labelling job to be used as the input to a machine learning model.
This feature will not help ensure the accuracy of worker annotations.
Option E is incorrect.
The Ground Truth Mechanical Turk feature gives you access to a large pool of labelling workers.
While increasing the number of workers at your disposal, this feature will not help ensure the accuracy of worker annotations.
Reference:
Please see the Amazon SageMaker developer guide titled Annotation Consolidation, and the Amazon Machine Learning blog titled Use the wisdom of crowds with Amazon SageMaker Ground Truth to annotate data more accurately, and GitHub repository titled Amazon Sagemaker Examples Introduction to Ground Truth Labeling Jobs.
Amazon Ground Truth is a fully managed data labelling service that enables you to efficiently and accurately label datasets. The annotation consolidation feature of Amazon Ground Truth helps improve the quality of labelled data by consolidating annotations from multiple human labelers into a single, accurate label. This is particularly important for tasks where accuracy is paramount, such as in the medical imagery domain.
To ensure the accuracy of the labelling tasks, you should use two annotation consolidation functions: Bounding box and Named entity.
Bounding box: The bounding box annotation type is used to identify the location of an object in an image. This annotation type is commonly used for object detection tasks where the goal is to identify the location of a specific object within an image. In the context of medical imagery, bounding boxes can be used to identify specific regions of interest within an image such as a tumor, lesion, or anatomical structure. Using the annotation consolidation feature for bounding box annotations, multiple labelers' annotations can be combined to produce a final, accurate bounding box annotation.
Named entity: Named entity annotation type is used to label named entities in text such as the names of people, organizations, locations, or medical terms. In the context of medical imagery, named entity annotations can be used to label anatomical structures or medical conditions. For example, a labeler could label a region of an image as "lung," "liver," or "tumor." Using the annotation consolidation feature for named entity annotations, multiple labelers' annotations can be combined to produce a final, accurate named entity annotation.
The other options provided in the question are not relevant to the task at hand. Semantic segmentation is a type of annotation that labels each pixel in an image, which is not necessary for this task. Output manifest is not an annotation type, but rather a file that describes the location and format of the output data. Mechanical Turk is a platform for sourcing human labelers, but not an annotation type or consolidation function.
In summary, to ensure the accuracy of labelling medical images for auto diagnostic software, the annotation consolidation feature of Ground Truth should be used with bounding box and named entity annotation types.
