Possible Cause of Deteriorating Model Accuracy | PMLE Exam Answer
Question
You built and manage a production system that is responsible for predicting sales numbers.
Model accuracy is crucial, because the production model is required to keep up with market changes.
Since being deployed to production, the model hasn't changed; however the accuracy of the model has steadily deteriorated.
What issue is most likely causing the steady decline in model accuracy?
Answers
Explanations
Click on the arrows to vote for the correct answer
A. B. C. D.D.
The most likely issue causing the steady decline in model accuracy in this scenario is option B, "Lack of model retraining."
There are a few reasons why this may be the case. First, the market is constantly changing, so the data used to train the model may no longer be representative of the current state of the market. This could be due to changes in consumer behavior, new products or competitors entering the market, or other factors that may not have been present when the model was originally trained. As a result, the model's predictions may no longer be accurate.
Second, even if the data used to train the model is still representative of the market, the model itself may be becoming outdated. Over time, models can become less effective as new techniques and algorithms are developed, and as the data they are applied to changes. Retraining the model on new data and with updated techniques can help ensure that it continues to make accurate predictions.
It's worth noting that poor data quality (option A) or an incorrect data split ratio during model training (option D) could also potentially contribute to declining model accuracy. For example, if the data used to train the model is noisy or contains errors, this could lead to inaccurate predictions. Similarly, if the model was initially evaluated on a test set that was not representative of the data it would be applied to in production, this could also lead to declining accuracy over time. However, in the scenario described in the question, it seems more likely that the issue is related to the need for model retraining, given the emphasis on the need for the model to keep up with market changes.
Finally, it's less likely that the issue is related to the number of layers in the model (option C). While the architecture of the model can certainly impact its accuracy, it's unlikely that a model that was initially effective would suddenly become ineffective simply because it had too few layers. Other factors, such as changes in the data or the need for retraining, would likely be the primary drivers of declining accuracy.
