Microsoft Azure AI Search¶

The Microsoft Azure AI Search connection type enables the Azure AI Search integrations.

Authenticating to Azure AI Search¶

There are three ways to connect to Azure AI Search using Airflow.

Use token credentials i.e. add the credentials of a service principal (client_id, secret, tenant) to the Airflow connection.
Use an API key by setting it as the password, without a client_id. A query key is enough to read documents, writing them needs an admin key.
Fallback on DefaultAzureCredential when the connection has neither. This includes a mechanism to try different options to authenticate: Managed System Identity, environment variables, authentication through Azure CLI… Set managed_identity_client_id and workload_identity_tenant_id to use a specific managed identity.

With a service principal or DefaultAzureCredential the identity needs the Search Index Data Reader role on the search service, or Search Index Data Contributor to write documents.

Default Connection IDs¶

All hooks related to Microsoft Azure AI Search use azure_ai_search_default by default.

Configuring the Connection¶

Search Service Endpoint: Specify the endpoint of the search service, e.g. https://<service>.search.windows.net. https:// is added to a bare host name.
Azure Client ID (optional): Specify the client_id of the service principal. Leave it empty to authenticate with an API key or with DefaultAzureCredential.
Azure Secret or API Key (optional): Specify the secret of the service principal, or an API key when no client_id is set. It can be left out to fall back on DefaultAzureCredential.
Azure Tenant ID (optional): Specify the tenantId of the service principal.
Default Index (optional): Specify the index the hook works on when it is given none.
Batch Size (optional): Specify the batch_size: the number of documents the hook sends per write request, from 1 to 1000 (the default).
Managed Identity Client ID (optional): The client ID of a user-assigned managed identity. If provided with workload_identity_tenant_id, they’ll pass to DefaultAzureCredential.
Workload Identity Tenant ID (optional): ID of the application’s Microsoft Entra tenant. Also called its “directory” ID. If provided with managed_identity_client_id, they’ll pass to DefaultAzureCredential.

When specifying the connection in environment variable you should specify it using URI syntax.

Note that all components of the URI should be URL-encoded.

For example:

export AIRFLOW_CONN_AZURE_AI_SEARCH_DEFAULT='azure-ai-search://:api%20key@my-service.search.windows.net?index=my-index'