Allianz Financial Services (AFS) is a banking group offering end-to-end banking and financial solutions in South East Asia through its consumer banking, business banking, Islamic banking, investment finance and stock broking businesses as well as unit trust and asset administration, having served the financial community over the past five decades. AFS launched EMR cluster to support their big data analytics requirements.
AFS is looking at streaming dataflow engine that you can use to run real-time stream processing on high-throughput data sources and support event time semantics for out-of-order events, exactly-once semantics, backpressure control, and APIs optimized for writing both streaming and batch applications.
What EMR Hadoop eco-system component can fulfill this requirement? The component also needs to integrate with other AWS services like Kinesis Streams and Elasticsearch.
select 1 option.
Click on the arrows to vote for the correct answer
A. B. C. D.Answer : B.
Option A is incorrect -Hue (Hadoop User Experience) is an open-source, web-based, graphical user interface for use with Amazon EMR and Apache Hadoop.
Hue groups together several different Hadoop ecosystem projects into a configurable interface.
Amazon EMR has also added customizations specific to Hue in Amazon EMR.
Hue acts as a front-end for applications that run on your cluster, allowing you to interact with applications using an interface that may be more familiar or user-friendly.
The applications in Hue, such as the Hive and Pig editors, replace the need to log in to the cluster to run scripts interactively using each application's respective shell.
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hue.htmlOption B is correct - Apache Flink is a streaming dataflow engine that you can use to run real-time stream processing on high-throughput data sources.
Flink supports event time semantics for out-of-order events, exactly-once semantics, backpressure control, and APIs optimized for writing both streaming and batch applications.
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.htmlOption C is incorrect -Apache Phoenix is used for OLTP and operational analytics, allowing you to use standard SQL queries and JDBC APIs to work with an Apache HBase backing store.
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-phoenix.htmlOption D is incorrect -Apache Tez is a framework for creating a complex directed acyclic graph (DAG) of tasks for processing data.
In some cases, it is used as an alternative to Hadoop MapReduce.
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-tez.htmlThe EMR Hadoop eco-system component that can fulfill AFS's requirement of a streaming dataflow engine with event time semantics for out-of-order events, exactly-once semantics, backpressure control, and APIs optimized for writing both streaming and batch applications, as well as integration with other AWS services like Kinesis Streams and Elasticsearch, is Apache Flink.
Apache Flink is a distributed streaming dataflow engine that enables real-time stream processing on high-throughput data sources. It supports event time semantics for out-of-order events, ensuring that data is processed based on its timestamp rather than its arrival time. It also provides exactly-once semantics, which guarantees that each event is processed only once, even in the event of failures or retries. Additionally, Flink provides backpressure control, which helps to prevent system overloads by controlling the rate at which data is processed based on available system resources.
Flink also provides APIs that are optimized for writing both streaming and batch applications, making it a flexible and versatile tool for big data analytics. It supports integration with other AWS services like Kinesis Streams and Elasticsearch, allowing for easy ingestion and analysis of data from a variety of sources.
Apache Hue is a web-based interface for interacting with Hadoop, but it does not provide the real-time stream processing capabilities that AFS requires. Apache Phoenix is a SQL query engine for Hadoop, while Apache Tez is a data processing framework for Hadoop, but neither of these components offer the streaming dataflow engine capabilities that are needed for this use case.