airflow.providers.amazon.aws.sensors.kinesis_analytics

Module Contents

Classes

KinesisAnalyticsV2BaseSensor

General sensor behaviour for AWS Managed Service for Apache Flink.

KinesisAnalyticsV2StartApplicationCompletedSensor

Waits for AWS Managed Service for Apache Flink application to start.

KinesisAnalyticsV2StopApplicationCompletedSensor

Waits for AWS Managed Service for Apache Flink application to stop.
class airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2BaseSensor(application_name, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]

Bases: airflow.providers.amazon.aws.sensors.base_aws.AwsBaseSensor[airflow.providers.amazon.aws.hooks.kinesis_analytics.KinesisAnalyticsV2Hook]

General sensor behaviour for AWS Managed Service for Apache Flink.

Subclasses must set the following fields:

  • INTERMEDIATE_STATES

  • FAILURE_STATES

  • SUCCESS_STATES

  • FAILURE_MESSAGE

  • SUCCESS_MESSAGE

Parameters

  • application_name (str) – Application name.

  • deferrable (bool) – If True, the sensor will operate in deferrable mode. This mode requires aiobotocore module to be installed. (default: False, but can be overridden in config file by setting default_deferrable to True)

aws_hook_class[source]
ui_color = '#66c3ff'[source]
INTERMEDIATE_STATES: tuple[str, Ellipsis] = ()[source]
FAILURE_STATES: tuple[str, Ellipsis] = ()[source]
SUCCESS_STATES: tuple[str, Ellipsis] = ()[source]
FAILURE_MESSAGE = ''[source]
SUCCESS_MESSAGE = ''[source]
poke(context, **kwargs)[source]

Override when deriving this class.

class airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2StartApplicationCompletedSensor(*, application_name, max_retries=75, poke_interval=120, **kwargs)[source]

Bases: KinesisAnalyticsV2BaseSensor

Waits for AWS Managed Service for Apache Flink application to start.

See also

For more information on how to use this sensor, take a look at the guide: Wait for an Amazon Managed Service for Apache Flink Application to start

Parameters

  • application_name (str) – Application name.

  • deferrable – If True, the sensor will operate in deferrable mode. This mode requires aiobotocore module to be installed. (default: False, but can be overridden in config file by setting default_deferrable to True)

  • poke_interval (int) – Polling period in seconds to check for the status of the job. (default: 120)

  • max_retries (int) – Number of times before returning the current state. (default: 75)

  • aws_conn_id – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • region_name – AWS region_name. If not specified then the default boto3 behaviour is used.

  • verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

INTERMEDIATE_STATES: tuple[str, Ellipsis][source]
FAILURE_STATES: tuple[str, Ellipsis][source]
SUCCESS_STATES: tuple[str, Ellipsis][source]
FAILURE_MESSAGE = 'AWS Managed Service for Apache Flink application start failed.'[source]
SUCCESS_MESSAGE = 'AWS Managed Service for Apache Flink application started successfully'[source]
template_fields: collections.abc.Sequence[str][source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2StopApplicationCompletedSensor(*, application_name, max_retries=75, poke_interval=120, **kwargs)[source]

Bases: KinesisAnalyticsV2BaseSensor

Waits for AWS Managed Service for Apache Flink application to stop.

See also

For more information on how to use this sensor, take a look at the guide: Wait for an Amazon Managed Service for Apache Flink Application to stop

Parameters

  • application_name (str) – Application name.

  • deferrable – If True, the sensor will operate in deferrable mode. This mode requires aiobotocore module to be installed. (default: False, but can be overridden in config file by setting default_deferrable to True)

  • poke_interval (int) – Polling period in seconds to check for the status of the job. (default: 120)

  • max_retries (int) – Number of times before returning the current state. (default: 75)

  • aws_conn_id – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • region_name – AWS region_name. If not specified then the default boto3 behaviour is used.

  • verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

INTERMEDIATE_STATES: tuple[str, Ellipsis][source]
FAILURE_STATES: tuple[str, Ellipsis][source]
SUCCESS_STATES: tuple[str, Ellipsis][source]
FAILURE_MESSAGE = 'AWS Managed Service for Apache Flink application stop failed.'[source]
SUCCESS_MESSAGE = 'AWS Managed Service for Apache Flink application stopped successfully'[source]
template_fields: collections.abc.Sequence[str][source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?