Simplest Approach to Process and Query Real-Time Streaming Data using SQL | AWS Service Recommendation
Question
You want to create a stream processing solution to process and query real-time streaming data using a SQL-based solution.
You are looking for the simplest approach available that AWS provides. What AWS service should you use?
Answers
Explanations
Click on the arrows to vote for the correct answer
A. B. C. D.Answer: B.
Option A is INCORRECT because Amazon Kinesis Data Streams is not an AWS service to build analytics applications.
It is an ingestion service that provides data streams to consumers (e.g., analytic applications).
AWS documentation mentions that “Amazon Kinesis Data Streams (KDS) is a massively scalable and durable real-time data streaming service…
The data collected is available in milliseconds to enable real-time analytics use cases such as real-time dashboards, real-time anomaly detection, dynamic pricing, and more”.
Option B is CORRECT because AWS documentation (FAQ section) mentions that “If you want a fully managed solution and you want to use SQL to process the data from your data stream, you should use Kinesis Data Analytics”.
Option C is INCORRECT because Amazon Kinesis Client Library (KCL) is a service to process and query streaming data using SQL.
KCL is more complicated than AWS Kinesis Data Analytics.
AWS documentation mentions that “Kinesis Data Analytics uses the KCL to read data from streaming data sources as one part of your underlying application.
The service abstracts this from you, as well as many of the more complex concepts associated with using the KCL, such as checkpointing”.
Option D is INCORRECT because Amazon Kinesis Data Firehose is a service for loading data streams and not creating SQL-based applications.
Diagram: none.
References:
https://aws.amazon.com/kinesis/data-streams/ https://aws.amazon.com/kinesis/data-analytics/faqs/?nc=sn&loc=6 https://aws.amazon.com/kinesis/data-firehose/The AWS service that provides the simplest approach for processing and querying real-time streaming data using a SQL-based solution is Amazon Kinesis Data Analytics.
Amazon Kinesis Data Analytics is a fully-managed service that enables you to easily process and analyze streaming data with SQL without the need for any coding. It integrates with Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose, allowing you to easily process and query real-time streaming data.
Here's a brief overview of the other AWS services mentioned in the options:
- Amazon Kinesis Data Streams is a service for ingesting, processing, and storing real-time streaming data. It's a low-level service that requires custom coding for data processing and querying.
- Amazon Kinesis Client Library is a client library that allows you to easily consume and process data from Amazon Kinesis Data Streams using Java or Python. It's not a SQL-based solution and requires custom coding for data processing and querying.
- Amazon Kinesis Data Firehose is a fully-managed service for delivering real-time streaming data to destinations such as Amazon S3, Amazon Redshift, and Amazon Elasticsearch. It's not a SQL-based solution and requires custom coding for data processing and querying.
Therefore, the correct answer is B. Amazon Kinesis Data Analytics.
