You develop data engineering solutions for a company.
You must integrate the company's on-premises Microsoft SQL Server data with Microsoft Azure SQL Database. Data must be transformed incrementally.
You need to implement the data integration solution.
Which tool should you use to configure a pipeline to copy data?
Click on the arrows to vote for the correct answer
A. B. C. D.C
The Integration Runtime is a customer managed data integration infrastructure used by Azure Data Factory to provide data integration capabilities across different network environments.
A linked service defines the information needed for Azure Data Factory to connect to a data resource. We have three resources in this scenario for which linked services are needed:
-> On-premises SQL Server
-> Azure Blob Storage
-> Azure SQL database
Note: Azure Data Factory is a fully managed cloud-based data integration service that orchestrates and automates the movement and transformation of data. The key concept in the ADF model is pipeline. A pipeline is a logical grouping of Activities, each of which defines the actions to perform on the data contained in
Datasets. Linked services are used to define the information needed for Data Factory to connect to the data resources.
https://docs.microsoft.com/en-us/azure/machine-learning/team-data-science-process/move-sql-azure-adfThe correct answer is C. Use Azure Data Factory UI with Blob storage linked service as the source.
Explanation:
Azure Data Factory is a cloud-based data integration service that allows you to create, schedule, and manage data pipelines that move and transform data from various sources to various destinations.
To integrate on-premises Microsoft SQL Server data with Microsoft Azure SQL Database, you can use the Azure Data Factory UI to create a pipeline that copies data incrementally. This involves configuring the following components:
Linked Services: Linked services define the connection strings and credentials for the source and destination data stores. For this scenario, you need to create a linked service for the on-premises Microsoft SQL Server data and another linked service for Microsoft Azure SQL Database.
Dataset: Datasets define the structure of the data in the source and destination data stores. For this scenario, you need to create a dataset for the on-premises Microsoft SQL Server data and another dataset for Microsoft Azure SQL Database.
Pipeline: A pipeline is a collection of activities that define the data flow from the source to the destination. For this scenario, you need to create a pipeline that uses the dataset for the on-premises Microsoft SQL Server data as the source and the dataset for Microsoft Azure SQL Database as the destination. You also need to configure the pipeline to perform incremental data loads by specifying the appropriate settings.
Regarding the answer options:
A. Use the Copy Data tool with Blob storage linked service as the source: This option is incorrect because Blob storage is not the source for this scenario.
B. Use Azure PowerShell with SQL Server linked service as a source: This option is incorrect because PowerShell is not necessary for this scenario, and SQL Server is not the source.
C. Use Azure Data Factory UI with Blob storage linked service as the source: This option is correct because Azure Data Factory UI is the appropriate tool for creating the pipeline, and Blob storage is not the source.
D. Use the .NET Data Factory API with Blob storage linked service as the source: This option is incorrect because the .NET Data Factory API is not necessary for this scenario, and Blob storage is not the source.