Configuring Kinesis Agent for Streaming Log Files
Question
A company is planning on using the Kinesis agent to transfer several logs files from multiple EC2 Instances.
There are several log files from various locations that need to be streamed from the servers to Kinesis.
Which of the following do you need to configure to ensure that the data gets streamed accordingly?
Answers
Explanations
Click on the arrows to vote for the correct answer
A. B. C. D.Answer - A.
The AWS Documentation mentions the following.
########
Monitor Multiple File Directories and Write to Multiple Streams.
By specifying multiple flow configuration settings, you can configure the agent to monitor multiple file directories and send data to multiple streams.
In the following configuration example, the agent monitors two file directories and sends data to an Kinesis stream and a Kinesis Data Firehose delivery stream respectively.
Note that you can specify different endpoints for Kinesis Data Streams and Kinesis Data Firehose so that your Kinesis stream and Kinesis Data Firehose delivery stream don't need to be in the same region.
{
"cloudwatch.emitMetrics": true,
"kinesis.endpoint": "https://your/kinesis/endpoint",
"firehose.endpoint": "https://your/firehose/endpoint",
"flows": [
{
"filePattern": "/tmp/app1.log*",
"kinesisStream": "yourkinesisstream"
},
{
"filePattern": "/tmp/app2.log*",
"deliveryStream": "yourfirehosedeliverystream"
}
]
}
########
Option B is incorrect since the KCL will be used to read data from the streams and not the files.
Option C is incorrect since this is used on the consumer side.
Option D is incorrect since there is no mention on how long it would be required to maintain the records.
For more information on using the Kinesis agent, please refer to the below URL.
https://docs.aws.amazon.com/streams/latest/dev/writing-with-agents.htmlTo transfer log files from multiple EC2 instances to Kinesis, the Kinesis agent needs to be configured properly. The Kinesis agent is a pre-built Java application that can help to collect and send data to Kinesis. The agent is installed on the EC2 instances that will be used to send data to Kinesis.
Out of the given options, Option A is the most appropriate answer. To configure the Kinesis agent, we need to modify the agent configuration file. Specifically, we need to modify the "flows" section of the agent configuration file to ensure that the data gets streamed accordingly. The flows section specifies the source of the log data, the destination of the log data, and any optional data transformations that need to be applied before the data is sent to Kinesis.
Option B is incorrect because the Kinesis Client Library (KCL) is not used to read the different files. The KCL is a Java library that helps to consume data from a Kinesis stream.
Option C is incorrect because Enhanced Fan-Out is not directly related to configuring the Kinesis agent. Enhanced Fan-Out is a Kinesis feature that enables multiple consumers to read data from a single Kinesis stream with low latency and high throughput.
Option D is incorrect because changing the data retention period on the stream does not impact the configuration of the Kinesis agent. The data retention period on a Kinesis stream determines how long the data will be stored in the stream before it is automatically deleted.
Therefore, Option A is the correct answer, and the flows section of the agent configuration file needs to be modified to ensure that the data gets streamed accordingly.
