You started working on a classification problem with time series data and achieved an area under the receiver operating characteristic curve (AUC ROC) value of 99% for training data after just a few experiments.
You haven't explored using any sophisticated algorithms or spent any time on hyperparameter tuning.
What should your next step be to identify and fix the problem?
Click on the arrows to vote for the correct answer
A. B. C. D.B.
Given that the model has achieved an AUC ROC of 99% on the training data after just a few experiments, the first thing to suspect is overfitting. Overfitting occurs when the model is too complex and has learned to fit the training data too well, resulting in poor generalization to unseen data.
Therefore, the most appropriate step is to address overfitting by using a less complex algorithm. This could involve using a simpler model architecture or reducing the number of features in the dataset. This will help prevent the model from learning spurious patterns in the training data and improve its ability to generalize to unseen data.
Option B, applying nested cross-validation, is a technique used to prevent data leakage, which occurs when information from the test set leaks into the training set, resulting in overly optimistic performance estimates. While this is an important step in building a robust machine learning model, it does not address the issue of overfitting in this case.
Option C, removing highly correlated features, is another way to address overfitting, but it is not as effective as using a less complex algorithm. Additionally, removing features could result in loss of valuable information and reduce the performance of the model.
Option D, tuning the hyperparameters to reduce the AUC ROC value, is not an appropriate step to address overfitting. Hyperparameters are parameters that control the behavior of the model during training, and tuning them to reduce the performance of the model is not a solution to overfitting.
In summary, the best approach in this scenario is to address overfitting by using a less complex algorithm, which can help the model generalize better to unseen data.