Not Involved in Real-time Twitter Data Ingestion to Apache Spark Cluster with Azure Databricks

Correct Answer: D.

There are 3 major parts to this solution.

Firstly, we need to ingest data to Azure Event hubs, integrate it with Azure Databricks to process the messages, and finally think about accessing this data using Twitter API.

The following are the tasks involved in this setup as per Microsoft documentation.

Create an Azure Databricks workspace.

Create a Spark cluster in Azure Databricks.

Create a Twitter app to access streaming data.

Create notebooks in Azure Databricks.

Attach libraries for Event Hubs and Twitter API.

Send tweets to Event Hubs.

Read tweets from Event Hubs.

Options A, B, C and E are incorrect: These are the steps involved in achieving this solution.

Option D is correct: Notebooks should be created in Azure Databricks, not workspace.

To know more, please refer to the docs below:

The option that is NOT involved in the solution to ingest real-time Twitter data to Apache Spark Cluster using Azure Databricks is:

C. Create a Twitter app to access streaming data.

Explanation: To ingest real-time Twitter data to Apache Spark Cluster using Azure Databricks, the following steps need to be performed:

A. Attach libraries for Event Hubs and Twitter API: Azure Databricks provides pre-built libraries for Event Hubs and Twitter API, which need to be attached to the Databricks workspace to use them. Attach the libraries to the cluster by following the instructions in the Azure Databricks documentation.

B. Read tweets from Event Hubs: Create an Event Hub in Azure and configure it to receive data from Twitter. Stream the Twitter data into the Event Hub. Then, use the pre-built libraries for Event Hubs in Azure Databricks to read the tweets from the Event Hub.

C. Create a Twitter app to access streaming data: This option is not required as we are using Event Hubs to receive the streaming data from Twitter.

D. Create workspace in Azure Databricks: Create a workspace in Azure Databricks and set up the necessary security and access controls. This workspace will be used to manage the Spark cluster and notebooks used for processing the data.

E. Create a Spark cluster in Azure Databricks: Create a Spark cluster in Azure Databricks and attach the necessary libraries for processing the Twitter data. The cluster will be used to run Spark jobs on the data and extract insights.

Overall, the solution involves attaching libraries for Event Hubs and Twitter API, reading tweets from Event Hubs, creating a workspace in Azure Databricks, and creating a Spark cluster in Azure Databricks.