SageMaker Model Tracking: AWS SDK API Code Example

Linear Learner Model Search Results

Prev Question Next Question

Question

You work on an application development team for a new start-up social media site.

Your team is made up of data scientists and machine learning specialists, of which you are the lead machine learning specialist.

Your team has built a model in SageMaker using the built-in linear learner algorithm.

The team has performed several training runs to find the best datasets and hyperparameters.

You have decided to use the SageMaker model tracking capability to manage the many training runs your team has produced. You have asked your team to show you the results of their efforts to help you lead them in deciding which hyperparameters and test datasets to use.

They have used the AWS SDK API for SageMaker to produce the data for your decision.

The following is a section of code from their use of the SageMaker model tracking capability.

What does the code do? search_params = { “MaxResults”: 10, “Resource”: “TrainingJob”, “SearchExpression”: { “Filters”: [{ “Name”: “Tags.Model”, “Operator”: “Equals”, “Value”: ”Model_Social_Media_Classifier”, }]}, “SortBy”: “”Metrics.train:precision”, “SortOrder”: “Descending” } smclient = boto3.client(service_name='sagemaker') results = smclient.search(**search_params)

Answers

Explanations

Click on the arrows to vote for the correct answer

A. B. C. D.

Answer: C.

Option A is incorrect.

The code uses the SageMaker python client API to search your team's SageMaker resources (such as training run results) for a specific model's training run results.

It does not run any training jobs.

Option B is incorrect.

The code uses the SageMaker python client API to search your team's SageMaker resources (such as training run results) for a specific model's training run results.

It does not search for the best hyperparameters.

Option C is correct.

The code uses the SageMaker python client API to search your team's SageMaker resources (such as training run results) for a specific model's training run results.

It then sorts the results by the precision metric in descending order.

This will allow you to see which training model run performed the best from a precision perspective.

The results give that model's algorithm, data sources, hyperparameter values, and metrics results.

Option D is incorrect.

The code uses the SageMaker python client API to search your team's SageMaker resources (such as training run results) for a specific model's training run results.

It does not run any training jobs.

Reference:

Please see the AWS announcement titled New Model Tracking Capabilities for Amazon SageMaker Are Now Generally Available, the AWS Machine Learning blog titled Using model attributes to track your training runs on Amazon SageMaker, the Amazon SageMaker developer guide titled Search, the Amazon SageMaker developer guide titled Manage Machine Learning Experiments, the AWS SageMaker Client boto3 docs, and the Amazon SageMaker developer guide titled Tune a Linear Learner Model.

The code provided is using the SageMaker model tracking capability to search for training jobs based on certain criteria.

Let's break down the code:

python
search_params = {     “MaxResults”: 10,     “Resource”: “TrainingJob”,     “SearchExpression”: {         “Filters”: [{             “Name”: “Tags.Model”,             “Operator”: “Equals”,             “Value”: ”Model_Social_Media_Classifier”,         }]     },     “SortBy”: “”Metrics.train:precision”,     “SortOrder”: “Descending” }

The search_params dictionary contains the parameters for the search. Here's what each parameter does:

  • MaxResults: Specifies the maximum number of results to return. In this case, the maximum number of results is set to 10.
  • Resource: Specifies the resource type to search for. In this case, the resource type is set to TrainingJob.
  • SearchExpression: Contains a dictionary that specifies the search filters to apply.
  • Filters: Contains a list of filter expressions. In this case, there is only one filter expression.
  • Name: Specifies the name of the tag to filter on. In this case, the name is Tags.Model.
  • Operator: Specifies the operator to use for the filter. In this case, the operator is Equals.
  • Value: Specifies the value to match for the filter. In this case, the value is Model_Social_Media_Classifier.
  • SortBy: Specifies the metric to use for sorting the results. In this case, the metric is train:precision.
  • SortOrder: Specifies the order in which to sort the results. In this case, the order is Descending.

Now let's look at the next two lines of code:

python
smclient = boto3.client(service_name='sagemaker') results = smclient.search(**search_params)

The first line creates a SageMaker client using the boto3 library. The second line calls the search method on the SageMaker client, passing in the search_params dictionary as an argument.

Based on the above information, we can deduce that the code is searching for the 10 best training runs for a model that has been tagged as Model_Social_Media_Classifier, sorted by the precision metric in descending order. Therefore, the correct answer is C.

Prev Question Next Question