For analysis purposes, you need to send all the logs from all of your Compute Engine instances to a BigQuery dataset called platform-logs.
You have already installed the Cloud Logging agent on all the instances.
You want to minimize cost.
What should you do?
Click on the arrows to vote for the correct answer
A. B. C. D.C.
The correct answer is C: In Cloud Logging, create a filter to view only Compute Engine logs. Click Create Export. Choose BigQuery as Sink Service, and the platform-logs dataset as Sink Destination.
This answer is the most cost-effective option that will meet the requirements stated in the question. Let's break down why:
Option A involves giving the BigQuery Data Editor role to the service accounts used by your instances and updating the instances' metadata to add a logs-destination value. While this option would allow logs to be sent directly to BigQuery, it could be more costly because every log message sent to BigQuery is considered a billable event, and you could end up with a high number of unnecessary log messages. This is because every single log message would be sent to BigQuery, not just Compute Engine logs.
Option B involves creating a logs export with a Cloud Pub/Sub topic called logs as a sink, creating a Cloud Function that is triggered by messages in the logs topic, and configuring that Cloud Function to drop logs that are not from Compute Engine and to insert Compute Engine logs in the platform-logs dataset. This option is more complex and expensive because it involves multiple components, such as Cloud Pub/Sub, Cloud Functions, and the additional processing overhead needed to filter logs. Additionally, since the logs need to be filtered in a Cloud Function, every log message will be processed and billed, which could result in higher costs.
Option D involves creating a Cloud Function that has the BigQuery User role on the platform-logs dataset and configuring this Cloud Function to create a BigQuery Job that executes a specific query. This option is not ideal because it requires additional processing overhead to filter logs, and every log message will be processed and billed. Also, scheduling this function to run once a day using Cloud Scheduler can result in higher costs because every execution of the function is a billable event.
Option C is the most cost-effective solution as it involves creating a filter to view only Compute Engine logs and sending them directly to BigQuery. This option is efficient as it only sends Compute Engine logs to BigQuery, which avoids unnecessary costs. Additionally, it requires no additional processing overhead to filter logs, and there are no extra components required to process the logs, which keeps costs low.