Effective Ways to Optimize Shards for AWS Kinesis Stream Performance
Question
HikeHills.com (HH) is an online specialty retailer that sells clothing and outdoor refreshment gear for trekking, go camping, boulevard biking, mountain biking, rock hiking, ice mountaineering, skiing, avalanche protection, snowboarding, fly fishing, kayaking, rafting, road and trace running, and many more. HH runs their entire online infrastructure on java based web applications running on AWS.
The HH is capturing click stream data and use custom-build recommendation engine to recommend products which eventually improve sales, understand customer preferences and already using AWS kinesis KPL to collect events and transaction logs and process the stream. HH IT team identified lot of performance issues with the Kinesis Stream and based on the metrics captured, identified hot and cold shards.IT team wants to effectively improve the performance of the shards.
How can they achieve that? Select 2 options.
Answers
Explanations
Click on the arrows to vote for the correct answer
A. B. C. D.Answer: A, B.
Option A is correct.
Performance of a shard (hot shard) can be improve by splitting the shard based on the hash keys that target those shards.
https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-resharding-strategies.htmlOption B is correct.
Performance of a shard (more data) can be improve by splitting the shard based on the hash keys that target those shards.
https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-resharding-strategies.htmlOption C is incorrect.
Hot shards are split, cold shards are always merged to reduce unnecessary used capacity.
https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-resharding-strategies.htmlOption D is incorrect.
Hot shards are split, cold shards are always merged to reduce unnecessary used capacity.
https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-resharding-strategies.htmlHikeHills.com (HH) is an online retailer that captures clickstream data to recommend products to its customers. To process the clickstream data, they are using AWS Kinesis KPL, which collects events and transaction logs and processes the stream. The IT team has identified performance issues with the Kinesis Stream, and based on the metrics captured, they have identified hot and cold shards. The IT team wants to improve the performance of the shards.
AWS Kinesis Streams is a distributed stream processing service that can scale to handle a large volume of data. It is designed to handle large amounts of data in real-time and can process data from thousands of sources. A shard in Kinesis Streams is a unit of data throughput capacity that represents a fixed unit of capacity. Each shard has a fixed capacity of 1MB/sec or 1000 records/sec, whichever comes first.
To effectively improve the performance of the shards, the IT team can perform the following actions:
A. Split the hot shards to increase capacity for the hash keys that target those shards.
A hot shard is a shard that receives a high volume of data. The IT team can split the hot shards into smaller shards to increase the capacity for the hash keys that target those shards. This will distribute the workload among more shards, which will increase the throughput capacity and reduce the latency.
For example, if a hot shard is receiving 2MB/sec of data, it can be split into two shards of 1MB/sec capacity. This will double the capacity of the hot shard and improve the performance of the stream.
B. Split the shards that receive more data to increase capacity for the hash keys that target those shards.
The IT team can also split the shards that receive more data to increase the capacity for the hash keys that target those shards. This will increase the throughput capacity of the shard and reduce the latency.
For example, if a shard is receiving 1.5MB/sec of data, it can be split into two shards of 0.75MB/sec capacity. This will increase the capacity of the shard and improve the performance of the stream.
C. Split the cold shards to increase capacity for the hash keys that target those shards.
A cold shard is a shard that receives a low volume of data. Splitting the cold shards may not improve the performance of the stream because the capacity of the shard is not being fully utilized. However, if the IT team wants to increase the capacity of the cold shard, they can split it into smaller shards to increase the capacity for the hash keys that target those shards.
D. Split the shards that receive less data to increase capacity for the hash keys that target those shards.
Splitting the shards that receive less data may not improve the performance of the stream because the capacity of the shard is not being fully utilized. However, if the IT team wants to increase the capacity of the shard, they can split it into smaller shards to increase the capacity for the hash keys that target those shards.
In summary, the IT team can improve the performance of the Kinesis Stream by splitting the hot shards and the shards that receive more data to increase the capacity for the hash keys that target those shards.
