Azure Data Platform · Azure Logic Apps · Azure Services · Data Engineering · Real Scenario Logic's · SharePoint

File Movement from SharePoint to Azure Data Lake Storage Gen 2

Introduction

In this blogpost, we will see the how to do file movements in automatic way from SharePoint to ADLS Gen 2 via Azure Logic Apps. The use case, we are going to see here is when a person/someone uploads file in SharePoint(Source), we need to automatically move those files in the required Storage Account-ADLS Gen 2(Sink/Target) /container/folder/ with the same file name. There may be many possible ways to do the same, here we will see how to do this easily within 2 actions via Azure Logic Apps from Azure Portal.

Prerequisite

  • Specific SharePoint Folder/File level Upload/Read/Edit/Manage kind of Accesses.
  • Active Azure Subscription. If you don’t have, create a free account.
  • Subscription Level Contributor access or Owner access with basic understandings of using Azure and its components and knowing to create Simple blank Azure Logic App. we can check our Access in Subscriptions–>Access Control (IAM)–>View my Access. (like below)

SharePoint

SharePoint with Sample Path

ADLS Gen 2

ADLS Gen 2 in MS-Azure Storage Explorer with Sample Path

Azure Logic Apps

Create a Simple Blank Logic App and follow the below detailed step by step flows.

Azure Logic App with 2 Actions (Brief here & Detailed Steps below mentioned)

Detailed Step by Step Flows:

1. When a file is created in a folder

Type and choose ‘SharePoint
Choose Triggers and choose the above highlighted trigger
Under Parameters TAB, provide Site Address, Folder Id and How Often we need to check this & Keep the rest as default; create/change Connection if required

2. Create blob (V2)

Type and choose Azure Blob Storage
Create or Change Connection if required

Default Authentication Type is Access key. If so, we need to provide both Azure Storage Account name as well as Azure Storage Account’s respective Access Key. We can also use Azure AD Integrated option as Authentication type.

Under Parameters TAB, provide Storage account name, Folder path, Blob name (File name in suggesting expression list) and Blob Content (File Content in suggesting expression list) & Keep the rest as default

Unit Testing

Unit Testing by Placing File in SharePoint’s sample path and Checking the moved file in ADLS Gen 2’s sample path and Cross-Check the Run History in Azure Logic App

Additional Notes – which we will see in upcoming blogposts

We can also ‘Add an action‘s like below, to make it much more effective as well as automated:

  1. Adding ‘Create a pipeline run‘ to run the ADF Pipeline automatically(Once ADLS receives file from SharePoint).
  2. Adding ‘Send an email (V2)‘ to automate mailings that we will receive once the above steps are completed.

Summary

Thus, with the help of this blogpost we saw regarding the file movement in automatic way from SharePoint (Source) to ADLS Gen 2 (Target) with the help of Azure Logic Apps in easy and simple steps, just 2 actions (2 actions 2 do).

If you really like this blogpost, Please do Like, Share & Follow Blog and Show your Support for many more interesting upcoming Posts!

Advertisement

One thought on “File Movement from SharePoint to Azure Data Lake Storage Gen 2

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s