A well-architected serverless mobile application is being analyzed to get more insights into improving the user experience.
The business unit chief has gathered data scientists in the analytics and machine learning team to design SQL queries, dashboards and offer visualizations based on logs to transition to a decision-making stage.
You are the DevOps engineer assigned to discuss and support the solution architecture.
What are some of the important aspects of the log delivery pipeline to implement the architecture proposal?
Click on the arrows to vote for the correct answer
A. B. C. D.Correct Answer: A.
The key points in the question are SQL queries for analytics purposes based on logs and implementing a log delivery pipeline.
What is common in all answers is the use of Amazon Kinesis Data Firehose data transformation.
You can enable Amazon Kinesis Data Firehose data transformation when you create your delivery stream.
The DeliveryStreamType property indicates that the delivery stream delivers real-time streaming data to a destination.
The delivery stream type has to be set as DirectPut since it allows access to the delivery stream directly or to send records from CloudWatch Logs among other options.
Amazon Kinesis Data Firehose supports a Lambda invocation time of up to 5 minutes.
If your Lambda function takes more than 5 minutes to complete, you get the following error: Firehose encountered timeout errors when calling AWS Lambda.
The maximum supported function timeout is 5 minutes.
Incorrect Answers:
Option B is incorrect because Amazon Kinesis Data Firehose supports a Lambda invocation time of up to 5 minutes, not 30 minutes.
Option C is incorrect because the DeliveryStreamType property as KinesisStreamAsSource is used when the delivery stream uses a Kinesis data stream as a source.
This is not the case here.
Option D is incorrect because the DeliveryStreamType property as KinesisStreamAsSource is used when the delivery stream uses a Kinesis data stream as a source.
This is not the case here.
Besides, Amazon Kinesis Data Firehose supports a Lambda invocation time of up to 5 minutes, not 30 minutes.
References:
https://amzn.to/3fDcsla https://go.aws/3bkwIogThe question is asking about the important aspects of the log delivery pipeline to implement the architecture proposal for a well-architected serverless mobile application. The proposal involves analyzing logs to improve the user experience, and data scientists have been brought in to design SQL queries, dashboards, and visualizations. As the DevOps engineer, you are responsible for discussing and supporting the solution architecture.
The correct answer is A: Enable Amazon Kinesis Data Firehose data transformation. When creating the delivery stream, set the DeliveryStreamType property as DirectPut and the AWS CloudWatch Logs SubscriptionFilter destination as the delivery stream ARN. Amazon Kinesis Data Firehose invokes the specified Lambda function asynchronously to transform logs into JSON format records. Amazon Kinesis Data Firehose supports a Lambda invocation time of up to 5 minutes. An AWS Glue database and crawler components discover new transformations from AWS Lambda. Data scientists can run SQL-like queries on top of Amazon Athena and Amazon QuickSight for further analysis.
Here's why:
Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to destinations such as AWS S3, Amazon Redshift, and Amazon Elasticsearch Service. It can also transform the data before delivering it to the destination.
In this case, the proposal requires transforming logs into JSON format records, which can be accomplished using Kinesis Data Firehose data transformation. Setting the DeliveryStreamType property as DirectPut and specifying the AWS CloudWatch Logs SubscriptionFilter destination as the delivery stream ARN will allow Kinesis Data Firehose to receive log data from CloudWatch Logs and transform it using a Lambda function.
It's important to note that Kinesis Data Firehose supports a Lambda invocation time of up to 5 minutes, so the transformation must be designed accordingly.
The proposal also mentions that data scientists will be designing SQL queries, dashboards, and visualizations based on the logs. To facilitate this, an AWS Glue database and crawler components can be used to discover new transformations from the Lambda function. Finally, Amazon Athena and Amazon QuickSight can be used for further analysis, allowing data scientists to run SQL-like queries on the transformed log data.
Option B is incorrect because it incorrectly states that the DeliveryStreamType property should be set without specifying the destination as the delivery stream ARN. Option C is incorrect because it specifies KinesisStreamAsSource instead of DirectPut as the DeliveryStreamType property, which is not necessary in this case. Option D is incorrect because it specifies a longer Lambda invocation time of up to 30 minutes, which is not necessary and may result in unnecessary delays.