You work as a machine learning specialist for a small startup software company.
You are the only machine learning specialist in the company.
The founder of the company needs you to quickly build a machine learning model to test one of the team's minimum viable products with the intent of persevering or pivoting depending on the outcome of your model experiment.
You have decided to use SageMaker Autopilot to create your experiment.
You are creating your experiment in SageMaker Autopilot.
You have selected the S3 bucket, data source, and target feature for which to make predictions.
You are now ready to select the machine learning problem type and objective metric.
Which are viable combinations for your selections?
Click on the arrows to vote for the correct answer
A. B. C. D.Correct Answer: D.
Option A is incorrect.
When running a Multiclass Classification algorithm-based model, the only options that make sense in the SageMaker Autopilot available choices are Accuracy and F1macro.
Option B is incorrect.
When running a Regression algorithm-based model, the only option that makes sense in the SageMaker Autopilot available choices is MSE.Option C is incorrect.
When running a Binary Classification algorithm-based model, the only options that make sense in the SageMaker Autopilot available choices are F1, Accuracy, and AUC.Option D is correct.
When running a Regression algorithm-based model, the option that makes sense in the SageMaker Autopilot available choices is MSE.References:
Please see the Amazon SageMaker developer guide titled Automate model development with Amazon SageMaker Autopilot (https://docs.aws.amazon.com/sagemaker/latest/dg/autopilot-automate-model-development.html),
The AWS Getting Started Resource Center titled Create a machine learning model automatically (https://aws.amazon.com/getting-started/hands-on/create-machine-learning-model-automatically-sagemaker-autopilot/)
SageMaker Autopilot is a fully-managed service that helps to build, train, and deploy machine learning models quickly and easily. It automatically performs data exploration, feature engineering, model selection, and hyperparameter tuning, all while allowing the user to monitor and intervene in the process as needed.
When selecting the machine learning problem type and objective metric in SageMaker Autopilot, it is important to choose a viable combination that aligns with the business problem and the nature of the data. The following are the viable combinations among the given options:
A. Problem type: Multiclass Classification; Objective: AUC This combination is suitable when the problem involves predicting the probabilities of multiple classes. AUC (Area Under the Curve) is a popular metric to evaluate the performance of a multiclass classification model because it measures the model's ability to rank the predicted probabilities correctly.
B. Problem type: Regression; Objective: F1 This combination is not suitable because F1 is a metric that is commonly used for evaluating binary classification problems, not regression problems. F1 is the harmonic mean of precision and recall, and it measures how well the model balances precision and recall for positive class predictions. Therefore, this combination does not make sense.
C. Problem type: Binary Classification; Objective: MSE This combination is not suitable because MSE (Mean Squared Error) is a metric commonly used for evaluating regression problems, not binary classification problems. MSE measures the average squared difference between the predicted and actual target values, which does not align with the nature of binary classification problems.
D. Problem type: Regression; Objective: MSE. This combination is suitable when the problem involves predicting a continuous target variable. MSE is a commonly used metric for evaluating regression models because it measures the average squared difference between the predicted and actual target values, which aligns with the nature of regression problems.
In summary, the viable combinations for the selections are A and D.