airflow.providers.microsoft.azure.hooks.asb¶

Attributes¶

MessageCallback

Classes¶

`BaseAzureServiceBusHook`	BaseAzureServiceBusHook class to create session and create connection using connection string.
`AdminClientHook`	Interact with the ServiceBusAdministrationClient.
`MessageHook`	Interact with ServiceBusClient.

Module Contents¶

airflow.providers.microsoft.azure.hooks.asb.MessageCallback[source]¶

class airflow.providers.microsoft.azure.hooks.asb.BaseAzureServiceBusHook(azure_service_bus_conn_id=default_conn_name)[source]¶

Bases: airflow.hooks.base.BaseHook

BaseAzureServiceBusHook class to create session and create connection using connection string.

Parameters:: azure_service_bus_conn_id (str) – Reference to the Azure Service Bus connection.

conn_name_attr = 'azure_service_bus_conn_id'[source]¶

default_conn_name = 'azure_service_bus_default'[source]¶

conn_type = 'azure_service_bus'[source]¶

hook_name = 'Azure Service Bus'[source]¶

classmethod get_connection_form_widgets()[source]¶

Return connection widgets to add to connection form.

classmethod get_ui_field_behaviour()[source]¶

Return custom field behaviour.

conn_id = 'azure_service_bus_default'[source]¶

abstract get_conn()[source]¶: Return connection for the hook.

class airflow.providers.microsoft.azure.hooks.asb.AdminClientHook(azure_service_bus_conn_id=default_conn_name)[source]¶

Bases: BaseAzureServiceBusHook

Interact with the ServiceBusAdministrationClient.

This can create, update, list, and delete resources of a Service Bus namespace. This hook uses the same Azure Service Bus client connection inherited from the base class.

get_conn()[source]¶

Create a ServiceBusAdministrationClient instance.

This uses the connection string in connection details.

create_queue(queue_name, max_delivery_count=10, dead_lettering_on_message_expiration=True, enable_batched_operations=True)[source]¶

Create Queue by connecting to service Bus Admin client return the QueueProperties.

Parameters:

queue_name (str) – The name of the queue or a QueueProperties with name.
max_delivery_count (int) – The maximum delivery count. A message is automatically dead lettered after this number of deliveries. Default value is 10..
dead_lettering_on_message_expiration (bool) – A value that indicates whether this subscription has dead letter support when a message expires.
enable_batched_operations (bool) – Value that indicates whether server-side batched operations are enabled.

delete_queue(queue_name)[source]¶

Delete the queue by queue_name in service bus namespace.

Parameters:: queue_name (str) – The name of the queue or a QueueProperties with name.

create_topic(topic_name, azure_service_bus_conn_id='azure_service_bus_default', default_message_time_to_live=None, max_size_in_megabytes=None, requires_duplicate_detection=None, duplicate_detection_history_time_window=None, enable_batched_operations=None, size_in_bytes=None, filtering_messages_before_publishing=None, authorization_rules=None, support_ordering=None, auto_delete_on_idle=None, enable_partitioning=None, enable_express=None, user_metadata=None, max_message_size_in_kilobytes=None)[source]¶

Create a topic by connecting to service Bus Admin client.

Parameters:

topic_name (str) – Name of the topic.
default_message_time_to_live (datetime.timedelta | str | None) – ISO 8601 default message time span to live value. This is the duration after which the message expires, starting from when the message is sent to Service Bus. This is the default value used when TimeToLive is not set on a message itself. Input value of either type ~datetime.timedelta or string in ISO 8601 duration format like “PT300S” is accepted.
max_size_in_megabytes (int | None) – The maximum size of the topic in megabytes, which is the size of memory allocated for the topic.
requires_duplicate_detection (bool | None) – A value indicating if this topic requires duplicate detection.
duplicate_detection_history_time_window (datetime.timedelta | str | None) – ISO 8601 time span structure that defines the duration of the duplicate detection history. The default value is 10 minutes. Input value of either type ~datetime.timedelta or string in ISO 8601 duration format like “PT300S” is accepted.
enable_batched_operations (bool | None) – Value that indicates whether server-side batched operations are enabled.
size_in_bytes (int | None) – The size of the topic, in bytes.
filtering_messages_before_publishing (bool | None) – Filter messages before publishing.
authorization_rules (list[azure.servicebus.management.AuthorizationRule] | None) – List of Authorization rules for resource.
support_ordering (bool | None) – A value that indicates whether the topic supports ordering.
auto_delete_on_idle (datetime.timedelta | str | None) – ISO 8601 time span idle interval after which the topic is automatically deleted. The minimum duration is 5 minutes. Input value of either type ~datetime.timedelta or string in ISO 8601 duration format like “PT300S” is accepted.
enable_partitioning (bool | None) – A value that indicates whether the topic is to be partitioned across multiple message brokers.
enable_express (bool | None) – A value that indicates whether Express Entities are enabled. An express queue holds a message in memory temporarily before writing it to persistent storage.
user_metadata (str | None) – Metadata associated with the topic.
max_message_size_in_kilobytes (int | None) – The maximum size in kilobytes of message payload that can be accepted by the queue. This feature is only available when using a Premium namespace and Service Bus API version “2021-05” or higher. The minimum allowed value is 1024 while the maximum allowed value is 102400. Default value is 1024.

create_subscription(topic_name, subscription_name, lock_duration=None, requires_session=None, default_message_time_to_live=None, dead_lettering_on_message_expiration=True, dead_lettering_on_filter_evaluation_exceptions=None, max_delivery_count=10, enable_batched_operations=True, forward_to=None, user_metadata=None, forward_dead_lettered_messages_to=None, auto_delete_on_idle=None, filter_rule=None, filter_rule_name=None)[source]¶

Create a subscription with specified name on a topic and return the SubscriptionProperties for it.

An optional filter_rule can be provided to filter messages based on their properties. In particular, the correlation ID filter can be used to pair up replies to requests.

Parameters:

topic_name (str) – The topic that will own the to-be-created subscription.
subscription_name (str) – Name of the subscription that need to be created
lock_duration (datetime.timedelta | str | None) – ISO 8601 time span duration of a peek-lock; that is, the amount of time that the message is locked for other receivers. The maximum value for LockDuration is 5 minutes; the default value is 1 minute. Input value of either type ~datetime.timedelta or string in ISO 8601 duration format like “PT300S” is accepted.
requires_session (bool | None) – A value that indicates whether the queue supports the concept of sessions.
default_message_time_to_live (datetime.timedelta | str | None) – ISO 8601 default message time span to live value. This is the duration after which the message expires, starting from when the message is sent to Service Bus. This is the default value used when TimeToLive is not set on a message itself. Input value of either type ~datetime.timedelta or string in ISO 8601 duration format like “PT300S” is accepted.
dead_lettering_on_message_expiration (bool | None) – A value that indicates whether this subscription has dead letter support when a message expires.
dead_lettering_on_filter_evaluation_exceptions (bool | None) – A value that indicates whether this subscription has dead letter support when a message expires.
max_delivery_count (int | None) – The maximum delivery count. A message is automatically dead lettered after this number of deliveries. Default value is 10.
enable_batched_operations (bool | None) – Value that indicates whether server-side batched operations are enabled.
forward_to (str | None) – The name of the recipient entity to which all the messages sent to the subscription are forwarded to.
user_metadata (str | None) – Metadata associated with the subscription. Maximum number of characters is 1024.
forward_dead_lettered_messages_to (str | None) – The name of the recipient entity to which all the messages sent to the subscription are forwarded to.
auto_delete_on_idle (datetime.timedelta | str | None) – ISO 8601 time Span idle interval after which the subscription is automatically deleted. The minimum duration is 5 minutes. Input value of either type ~datetime.timedelta or string in ISO 8601 duration format like “PT300S” is accepted.
filter_rule (azure.servicebus.management.CorrelationRuleFilter | azure.servicebus.management.SqlRuleFilter | None) – Optional correlation or SQL rule filter to apply on the messages.
filter_rule_name (str | None) – Optional rule name to use applying the rule filter to the subscription
azure_service_bus_conn_id – Reference to the Azure Service Bus connection.

update_subscription(topic_name, subscription_name, max_delivery_count=None, dead_lettering_on_message_expiration=None, enable_batched_operations=None)[source]¶

Update an Azure ServiceBus Topic Subscription under a ServiceBus Namespace.

Parameters:

topic_name (str) – The topic that will own the to-be-created subscription.
subscription_name (str) – Name of the subscription that need to be created.
max_delivery_count (int | None) – The maximum delivery count. A message is automatically dead lettered after this number of deliveries. Default value is 10.
dead_lettering_on_message_expiration (bool | None) – A value that indicates whether this subscription has dead letter support when a message expires.
enable_batched_operations (bool | None) – Value that indicates whether server-side batched operations are enabled.

delete_subscription(subscription_name, topic_name)[source]¶

Delete a topic subscription entities under a ServiceBus Namespace.

Parameters:

subscription_name (str) – The subscription name that will own the rule in topic
topic_name (str) – The topic that will own the subscription rule.

class airflow.providers.microsoft.azure.hooks.asb.MessageHook(azure_service_bus_conn_id=default_conn_name)[source]¶

Bases: BaseAzureServiceBusHook

Interact with ServiceBusClient.

This acts as a high level interface for getting ServiceBusSender and ServiceBusReceiver.

get_conn()[source]¶

Create and returns ServiceBusClient by using the connection string in connection details.

send_message(queue_name, messages, batch_message_flag=False)[source]¶

Use ServiceBusClient Send to send message(s) to a Service Bus Queue.

By using batch_message_flag, it enables and send message as batch message.

Parameters:

queue_name (str) – The name of the queue or a QueueProperties with name.
messages (str | list[str]) – Message which needs to be sent to the queue. It can be string or list of string.
batch_message_flag (bool) – bool flag, can be set to True if message needs to be sent as batch message.

static send_list_messages(sender, messages)[source]¶

static send_batch_message(sender, messages)[source]¶

receive_message(queue_name, context, max_message_count=1, max_wait_time=None, message_callback=None)[source]¶

Receive a batch of messages at once in a specified Queue name.

Parameters:

queue_name (str) – The name of the queue name or a QueueProperties with name.
max_message_count (int | None) – Maximum number of messages in the batch.
max_wait_time (float | None) – Maximum time to wait in seconds for the first message to arrive.
message_callback (MessageCallback | None) – Optional callback to process each message. If not provided, then the message will be logged and completed. If provided, and throws an exception, the message will be abandoned for future redelivery.

receive_subscription_message(topic_name, subscription_name, context, max_message_count, max_wait_time, message_callback=None)[source]¶

Receive a batch of subscription message at once.

This approach is optimal if you wish to process multiple messages simultaneously, or perform an ad-hoc receive as a single call.

Parameters:

subscription_name (str) – The subscription name that will own the rule in topic
topic_name (str) – The topic that will own the subscription rule.
max_message_count (int | None) – Maximum number of messages in the batch. Actual number returned will depend on prefetch_count and incoming stream rate. Setting to None will fully depend on the prefetch config. The default value is 1.
max_wait_time (float | None) – Maximum time to wait in seconds for the first message to arrive. If no messages arrive, and no timeout is specified, this call will not return until the connection is closed. If specified, an no messages arrive within the timeout period, an empty list will be returned.