Azure Batch Operator¶

AzureBatchOperator¶

Use the AzureBatchOperator to trigger a task on Azure Batch

Below is an example of using this operator to trigger a task on Azure Batch

tests/system/microsoft/azure/example_azure_batch_operator.py[source]

    azure_batch_operator = AzureBatchOperator(
        task_id="azure_batch",
        batch_pool_id=POOL_ID,
        batch_pool_vm_size="standard_d2s_v3",
        batch_job_id="example-job",
        batch_task_command_line="/bin/bash -c 'set -e; set -o pipefail; echo hello world!; wait'",
        batch_task_id="example-task",
        vm_node_agent_sku_id="batch.node.ubuntu 22.04",
        vm_publisher="Canonical",
        vm_offer="0001-com-ubuntu-server-jammy",
        vm_sku="22_04-lts-gen2",
        target_dedicated_nodes=1,
    )

Below is an example of using this operator to trigger a task on Azure Batch with the deferrable flag, so that polling for job/task completion occurs on the Airflow Triggerer.

tests/system/microsoft/azure/example_azure_batch_operator.py[source]

azure_batch_operator_deferrable = AzureBatchOperator(
    task_id="azure_batch_deferrable",
    batch_pool_id=POOL_ID,
    batch_pool_vm_size="standard_d2s_v3",
    batch_job_id="example-job",
    batch_task_command_line="/bin/bash -c 'set -e; set -o pipefail; echo hello world!; wait'",
    batch_task_id="example-task",
    vm_node_agent_sku_id="batch.node.ubuntu 22.04",
    vm_publisher="Canonical",
    vm_offer="0001-com-ubuntu-server-jammy",
    vm_sku="22_04-lts-gen2",
    target_dedicated_nodes=1,
    deferrable=True,
)

Reference¶

For further information, look at:

Azure Batch Documentation