5.App Registration
To connect Azure Databricks to Azure services using an Azure App Registration and a service principal, you can follow these steps:
Create an App Registration:
Go to the Azure Portal.
Navigate to "Azure Active Directory" "App registrations" "New registration".
Fill in the required details and register the application.
Generate Client Secret:
After registration, go to "Certificates & secrets" under the newly created app registration.
Create a new client secret and note down the value. This secret will be used as the client secret when authenticating.
Assign Required Permissions:
Under the app registration, navigate to "API permissions".
Assign the necessary permissions based on the Azure services you want to connect to (e.g., Azure Storage, Azure SQL Database, etc.).
Retrieve Tenant ID, Application ID, and Client Secret:
Note down the "Directory (tenant) ID" from the app registration overview.
Note down the "Application (client) ID" from the app registration overview.
You already have the client secret generated in the previous step.
Create Azure Databricks Secret Scope:
Open your Azure Databricks workspace.
Go to the "Secrets" tab.
Create a new secret scope and set the scope name.
Create Secret in Azure Databricks:
Inside the secret scope, add secrets for clientId, clientSecret, directoryId, and resourceUrl (e.g., for Azure Storage, use https://storage.azure.com).
Set the corresponding values based on your app registration.
Access Secrets in Databricks Notebook:
In your Databricks notebook, you can access these secrets using %fs commands or Databricks Utilities.
python
Copy code
Access secrets using %fs command
dbutils.fs.mounts()
Access secrets using Databricks Utilities
By following these steps, you can securely connect Azure Databricks to Azure services using an Azure App Registration, a service principal, and Azure Databricks secret scopes.