Prevent PII Leakage in Stackdriver Logs
Question
You are running an application on Compute Engine and collecting logs through Stackdriver.
You discover that some personally identifiable information (PII) is leaking into certain log entry fields.
You want to prevent these fields from being written in new log entries as quickly as possible.
What should you do?
Answers
Explanations
Click on the arrows to vote for the correct answer
A. B. C. D.B.
Reference - https://cloud.google.com/logging/docs/agent/logging/configuration.
The best solution to prevent PII from leaking into certain log entry fields in Stackdriver is to remove the fields from the log entries in flight. This can be accomplished by using a Fluentd filter plugin or output plugin.
A. The filter-record-transformer Fluentd filter plugin can be used to remove the fields from the log entries in flight. This plugin can be configured to filter and transform the log records before they are written to Stackdriver Logging. With this plugin, you can specify a list of fields to remove from the log entries.
B. The fluent-plugin-record-reformer Fluentd output plugin can also be used to remove the fields from the log entries in flight. This plugin can be configured to modify the structure of log records, including removing certain fields. With this plugin, you can specify the fields to remove from the log entries.
C. Waiting for the application developers to patch the application is not a recommended solution because it could take time for the developers to release a patch, and the PII would continue to be exposed until the patch is released. Additionally, this solution does not address the existing log entries that contain PII.
D. Staging log entries to Cloud Storage and then triggering a Cloud Function to remove the fields and write the entries to Stackdriver via the Stackdriver Logging API is a viable solution, but it is not as efficient as using a Fluentd filter or output plugin. This solution requires additional infrastructure and processing, and it could introduce additional latency to the logging process.
In summary, the best solution to prevent PII from leaking into certain log entry fields in Stackdriver is to use a Fluentd filter or output plugin to remove the fields from the log entries in flight.
