A large financial application generates logs in a comma-separated format and saves them to S3 for later processing.
After each file is generated, a message is sent to an SQS queue.
EC2 instances monitoring the SQS queue are running behind a load balancer in an Auto Scaling group.
An EC2 instance will pick up the message from SQS and start processing it.
Once the processing is completed, the processed files are stored into another S3 bucket to be used later in a reporting process.
The EC2 instances load their application from S3 on startup.
The application release logs show that there have been many updates to the application in the last month.
The data security and monitoring team wants to check the application logs to ensure that the logs do not contain any sensitive information.
How can you complete this process effectively without interrupting the Auto Scaling or the application release cycle?
Click on the arrows to vote for the correct answer
A. B. C. D.Correct Answer: C because Cloudwatch logs provide information on the application behavior.
A Lambda function can process CloudWatch Logs.
Option A is INCORRECT because the CloudTrail only monitors the API calls made to AWS resources and does not record the application logs.
Option B is INCORRECT because suspending the ASG terminate Instances is not effective and may affect the application release cycle during high utilization period.
Administrators should be able to get the logs without even log in to EC2 Instances.
Option D is INCORRECT because this method is not straight-forward or cost-effective.
Option A: Enable CloudTrail and redirect all the system logs to an S3 bucket. Download the log files from the S3 bucket and check for sensitive information.
CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS account. It records all the API calls made within your account and delivers log files to an S3 bucket that you specify. In this scenario, enabling CloudTrail and directing all the system logs to an S3 bucket would ensure that all the API calls made to the S3 buckets are recorded. This would include the logs generated by the financial application. Once the logs are recorded, they can be downloaded and checked for sensitive information. This solution does not interrupt the Auto Scaling or the application release cycle.
Option B: Suspend the Auto Scaling termination process, and then log into the machines that started recently and check the logs.
This option requires suspending the Auto Scaling termination process, which means that new EC2 instances will not be launched if the number of instances falls below the desired capacity. This could affect the availability of the application if the suspended process is not resumed promptly. Additionally, logging into the machines that started recently would not be effective in this scenario since the EC2 instances are launched and terminated by the Auto Scaling group.
Option C: Install the CloudWatch Logs Agent and publish the logs to CloudWatch. Trigger a Lambda function to process the logs.
CloudWatch is a monitoring service for AWS resources and applications. It can collect and track metrics, collect and monitor log files, and set alarms. In this option, installing the CloudWatch Logs Agent on the EC2 instances and publishing the logs to CloudWatch would allow the logs to be monitored in real-time. A Lambda function can be triggered to process the logs and check for sensitive information. This option does not interrupt the Auto Scaling or the application release cycle.
Option D: Take daily snapshots of the EC2 instance volumes, and then mount the recent snapshots to another instance and check the logs.
This option requires taking daily snapshots of the EC2 instance volumes, which could result in a large number of snapshots being created, leading to increased storage costs. Additionally, mounting the recent snapshots to another instance would require stopping the original instance, which would result in downtime for the application. This option is not practical in this scenario since it requires stopping the EC2 instances, which would interrupt the Auto Scaling and the application release cycle.
Based on the given scenario, option A or option C would be the best solutions since they do not interrupt the Auto Scaling or the application release cycle. Option A involves enabling CloudTrail and redirecting all system logs to an S3 bucket, which can be downloaded and checked for sensitive information. Option C involves installing the CloudWatch Logs Agent and publishing the logs to CloudWatch, which can be monitored in real-time, and a Lambda function can be triggered to process the logs.