Azure Data lake Gen 2 · Azure Data Platform · Azure Databricks · Azure Services · Data Engineering · Python

Checking ADLS Gen 2 accesses in Azure Databricks via Service principal

Introduction

In this blogpost, we will see how easily we can test the Service Principal accesses like below for files and folders at ADLS gen 2 in dynamic way (within some secs) via re-runnable python script in notebook of Azure Databricks. We can also utilize the below code to test the working of Key Vaults as well.

  • Write
  • Copy
  • Read
  • Create Directory
  • Move
  • Remove

Check Service Principal Creation for Accessing ADLS Gen 2 in Azure Databricks regarding Service Principal Creation and reach GitHub for Azure Databricks Key vault creating secret Scope.

Provide the inputs like below and test the Service principal access

  • ContainerName
  • StorageAccountName
  • Folder, want to test the accesses
  • ScopeName, the name of the scope provided in Azure Databricks while linking Key Vault
  • ServicePrincipalID, the respective Key Vault Name
  • ServicePrincipalKey, the respective Key Vault Name
  • TenantID, the respective Key Vault Name

In the Setup variables, we can provide all the necessary checking 7 inputs within double quotes.

Once the Setup variables are set, then the connections can be build based on those.

We can easily cross check connecting via listing files/folders options{dbutils.fs.ls}.

In the above script,

  1. check1.csv file would be created using write {dbutils.fs.put} with 3 columns like name, dept and email.
  2. then, we can copy {dbutils.fs.cp} check1.csv file and create check2.csv file based on it.
  3. then, we can read {dbutils.fs.head} our check1.csv file.
  4. then, we can also create folder/directory {dbutils.fs.mkdirs} named subfolder.
  5. then we can move {dbutils.fs.mv} check2.csv file from parent folder to child folder called subfolder.
  6. finally, we are checking the removing access by removing {dbutils.fs.rm} check1.csv file in parent folder path.

If you are interest to re-use or practice the above python script/code, visit Arulmouzhi/AzureStuffs Github repos.

Recent Related Posts

Conclusion

Thus, in this blogpost, we saw how easily we can able to test the Service Principal accesses like write, copy, read, create directory, move, remove the files and folders at ADLS gen 2 in dynamic way using re-runnable python script in notebook of Azure Databricks.

The Special Thanks to my colleague Roosevelt who was behind the reasons for this post!

Follow Blog and Show your Support for many more interesting upcoming Posts!

Advertisement

One thought on “Checking ADLS Gen 2 accesses in Azure Databricks via Service principal

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s