airflow.providers.microsoft.azure.hooks.batch

Classes

AzureBatchHook

Hook for Azure Batch APIs.

Module Contents

class airflow.providers.microsoft.azure.hooks.batch.AzureBatchHook(azure_batch_conn_id=default_conn_name)[source]

Bases: airflow.hooks.base.BaseHook

Hook for Azure Batch APIs.

Parameters:

azure_batch_conn_id (str) – Azure Batch connection id of a service principal which will be used to start the container instance.

conn_name_attr = 'azure_batch_conn_id'[source]
default_conn_name = 'azure_batch_default'[source]
conn_type = 'azure_batch'[source]
hook_name = 'Azure Batch Service'[source]
classmethod get_connection_form_widgets()[source]

Return connection widgets to add to connection form.

classmethod get_ui_field_behaviour()[source]

Return custom field behaviour.

conn_id = 'azure_batch_default'[source]
property connection: azure.batch.BatchServiceClient[source]

Get the Batch client connection (cached).

get_conn()[source]

Get the Batch client connection.

Returns:

Azure Batch client

Return type:

azure.batch.BatchServiceClient

configure_pool(pool_id, vm_size, vm_node_agent_sku_id, vm_publisher=None, vm_offer=None, sku_starts_with=None, vm_sku=None, vm_version=None, os_family=None, os_version=None, display_name=None, target_dedicated_nodes=None, use_latest_image_and_sku=False, **kwargs)[source]

Configure a pool.

Parameters:

  • pool_id (str) – A string that uniquely identifies the Pool within the Account

  • vm_size (str) – The size of virtual machines in the Pool.

  • display_name (str | None) – The display name for the Pool

  • target_dedicated_nodes (int | None) – The desired number of dedicated Compute Nodes in the Pool.

  • use_latest_image_and_sku (bool) – Whether to use the latest verified vm image and sku

  • vm_publisher (str | None) – The publisher of the Azure Virtual Machines Marketplace Image. For example, Canonical or MicrosoftWindowsServer.

  • vm_offer (str | None) – The offer type of the Azure Virtual Machines Marketplace Image. For example, UbuntuServer or WindowsServer.

  • sku_starts_with (str | None) – The start name of the sku to search

  • vm_sku (str | None) – The name of the virtual machine sku to use

  • vm_version (str | None) – The version of the virtual machine

  • vm_version – str

  • vm_node_agent_sku_id (str) – The node agent sku id of the virtual machine

  • os_family (str | None) – The Azure Guest OS family to be installed on the virtual machines in the Pool.

  • os_version (str | None) – The OS family version

create_pool(pool)[source]

Create a pool if not already existing.

Parameters:

pool (azure.batch.models.PoolAddParameter) – the pool object to create

wait_for_all_node_state(pool_id, node_state)[source]

Wait for all nodes in a pool to reach given states.

Parameters:

  • pool_id (str) – A string that identifies the pool

  • node_state (set) – A set of batch_models.ComputeNodeState

configure_job(job_id, pool_id, display_name=None, **kwargs)[source]

Configure a job for use in the pool.

Parameters:

  • job_id (str) – A string that uniquely identifies the job within the account

  • pool_id (str) – A string that identifies the pool

  • display_name (str | None) – The display name for the job

create_job(job)[source]

Create a job in the pool.

Parameters:

job (azure.batch.models.JobAddParameter) – The job object to create

configure_task(task_id, command_line, display_name=None, container_settings=None, **kwargs)[source]

Create a task.

Parameters:

  • task_id (str) – A string that identifies the task to create

  • command_line (str) – The command line of the Task.

  • display_name (str | None) – A display name for the Task

  • container_settings – The settings for the container under which the Task runs. If the Pool that will run this Task has containerConfiguration set, this must be set as well. If the Pool that will run this Task doesn’t have containerConfiguration set, this must not be set.

add_single_task_to_job(job_id, task)[source]

Add a single task to given job if it doesn’t exist.

Parameters:

  • job_id (str) – A string that identifies the given job

  • task (azure.batch.models.TaskAddParameter) – The task to add

wait_for_job_tasks_to_complete(job_id, timeout)[source]

Wait for tasks in a particular job to complete.

Parameters:

  • job_id (str) – A string that identifies the job

  • timeout (int) – The amount of time to wait before timing out in minutes

test_connection()[source]

Test a configured Azure Batch connection.

Was this entry helpful?