Setting up an Azure Integration

The Azure integration uses DefaultAzureCredential for authentication, which supports multiple authentication methods. This guide focuses on setting up authentication using Azure CLI and Service Principal, which is recommended for production use.

Set up a Service Principal

Syncing from single subscription

You will need to create a service principal for the plugin to use:

Install the Azure CLI if you haven’t already
Open your terminal and run:

# Login to Azure
az login

Create a service principal with a Reader access:

# Register the security provider
az provider register --namespace 'Microsoft.Security'

# Create a service principal and grant Reader access
az ad sp create-for-rbac --name cloudquery-sp \
          --scopes /subscriptions/{subscription-id} --role Reader

The command will output credentials in the following format:

{
  "appId": "YOUR_AZURE_CLIENT_ID",
  "displayName": "cloudquery-sp",
  "password": "YOUR_AZURE_CLIENT_SECRET",
  "tenant": "YOUR_AZURE_TENANT_ID"
}

Save these credentials - you’ll need them in the next step.

Syncing from multiple subscriptions

There are two main approaches to sync data from multiple Azure subscriptions:

Management Group Level Access

The most dynamic approach is to scope the service principal at the Management Group level. This allows automatic discovery of all subscriptions under the specified Management Group(s), including any new subscriptions added later.

To create a service principal with Management Group access:

# Register the security provider
az provider register --namespace 'Microsoft.Security'

# Create service principal with Management Group access
az ad sp create-for-rbac --name cloudquery-sp-root-1 \
  --scopes /providers/Microsoft.Management/managementGroups/{management-group-name} \
  --role Reader

Specific Subscriptions Access

If you prefer to limit access to specific subscriptions, you can list them explicitly in the scope. This command grants access to all currently accessible subscriptions:

# Register the security provider
az provider register --namespace 'Microsoft.Security'

# Create service principal with access to specific subscriptions
az ad sp create-for-rbac --name cloudquery-sp \
  --scopes $(az account subscription list --query "[].id" -o tsv --only-show-errors | xargs) \
  --role Reader

⚠️

When using the specific subscriptions approach, the service principal won’t automatically get access to newly added subscriptions. You’ll need to run the command again to include new subscriptions.

Setting up a Azure Integration

In CloudQuery Platform, go to Data Pipelines → Integrations. Click Create Integration and type Azure to find the Azure integration.

Find Azure integration

Copy-paste saved credentials into the Value fields:

AZURE_TENANT_ID: <tenant from the output>
AZURE_CLIENT_ID: <appId from the output>
AZURE_CLIENT_SECRET: <password from the output>

Azure Yaml Configuration with secrets

You may want to make further adjustments to your YAML file, according to your requirements. For more information, see the Azure Integration Documentation
Click Test Connection to verify the configuration

Next Steps

With your Azure integration created, you can now proceed to use it in a new sync. This will give you the opportunity to specify when your Azure sync should be run, and to which destination databases.

GCP GitHub