Implementing an Azure Data Solution: Loading Data from Azure Data Lake Gen 2 Storage into Synapse Analytics

Loading Data from Azure Data Lake Gen 2 Storage into Synapse Analytics

Prev Question Next Question

Question

Note: This question is a part of series of questions that present the same scenario. Each question in the series contains a unique solution. Determine whether the solution meets the stated goals.

You develop a data ingestion process that will import data to an enterprise data warehouse in Azure Synapse Analytics. The data to be ingested resides in parquet files stored in an Azure Data Lake Gen 2 storage account.

You need to load the data from the Azure Data Lake Gen 2 storage account into the Data Warehouse.

Solution:

1. Create an external data source pointing to the Azure storage account

2. Create an external file format and external table using the external data source

3. Load the data using the INSERT'SELECT statement

Does the solution meet the goal?

Answers

Explanations

Click on the arrows to vote for the correct answer

A. B.

B

You load the data using the CREATE TABLE AS SELECT statement.

https://docs.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-load-from-azure-data-lake-store

The solution presented is a valid approach to load data from parquet files stored in an Azure Data Lake Gen 2 storage account into an enterprise data warehouse in Azure Synapse Analytics.

Here is a detailed explanation of the solution:

  1. Create an external data source pointing to the Azure storage account: An external data source is created to define the connection properties and credentials required to access the Azure Data Lake Gen 2 storage account. The data source specifies the storage account name and the account access key or a managed identity. This step is necessary to establish a connection between the data warehouse and the data lake.

  2. Create an external file format and external table using the external data source: An external file format is created to specify the data format and structure of the parquet files stored in the data lake. This format must match the data structure of the files to be ingested. An external table is then created using the same external data source and external file format. The external table defines the schema of the parquet files and allows them to be queried as if they were local tables in the data warehouse.

  3. Load the data using the INSERT'SELECT statement: Finally, the data is loaded from the external table into the target table in the data warehouse using an INSERT...SELECT statement. This statement copies the data from the external table to the target table, transforming and mapping the columns as necessary.

Overall, this solution meets the goal of loading data from an Azure Data Lake Gen 2 storage account into an enterprise data warehouse in Azure Synapse Analytics. Therefore, the answer is A. Yes.

Prev Question Next Question