airflow.providers.amazon.aws.operators.kinesis_analytics

Module Contents

Classes

KinesisAnalyticsV2CreateApplicationOperator

Creates an AWS Managed Service for Apache Flink application.

KinesisAnalyticsV2StartApplicationOperator

Starts an AWS Managed Service for Apache Flink application.

KinesisAnalyticsV2StopApplicationOperator

Stop an AWS Managed Service for Apache Flink application.
class airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2CreateApplicationOperator(application_name, runtime_environment, service_execution_role, create_application_kwargs=None, application_description='Managed Service for Apache Flink application created from Airflow', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.kinesis_analytics.KinesisAnalyticsV2Hook]

Creates an AWS Managed Service for Apache Flink application.

See also

For more information on how to use this operator, take a look at the guide: Create an Amazon Managed Service for Apache Flink Application

Parameters

  • application_name (str) – The name of application. (templated)

  • runtime_environment (str) – The runtime environment for the application. (templated)

  • service_execution_role (str) – The IAM role used by the application to access services. (templated)

  • create_application_kwargs (dict[str, Any] | None) – Create application extra properties. (templated)

  • application_description (str) – A summary description of the application. (templated)

  • aws_conn_id – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • region_name – AWS region_name. If not specified then the default boto3 behaviour is used.

  • verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

aws_hook_class[source]
ui_color = '#44b5e2'[source]
template_fields: collections.abc.Sequence[str][source]
template_fields_renderers: ClassVar[dict][source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2StartApplicationOperator(application_name, run_configuration=None, wait_for_completion=True, waiter_delay=60, waiter_max_attempts=20, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.kinesis_analytics.KinesisAnalyticsV2Hook]

Starts an AWS Managed Service for Apache Flink application.

See also

For more information on how to use this operator, take a look at the guide: Start an Amazon Managed Service for Apache Flink Application

Parameters

  • application_name (str) – The name of application. (templated)

  • run_configuration (dict[str, Any] | None) – Application properties to start Apache Flink Job. (templated)

  • wait_for_completion (bool) – Whether to wait for job to stop. (default: True)

  • waiter_delay (int) – Time in seconds to wait between status checks. (default: 60)

  • waiter_max_attempts (int) – Maximum number of attempts to check for job completion. (default: 20)

  • deferrable (bool) – If True, the operator will wait asynchronously for the job to stop. This implies waiting for completion. This mode requires aiobotocore module to be installed. (default: False)

  • aws_conn_id – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • region_name – AWS region_name. If not specified then the default boto3 behaviour is used.

  • verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

aws_hook_class[source]
ui_color = '#44b5e2'[source]
template_fields: collections.abc.Sequence[str][source]
template_fields_renderers: ClassVar[dict][source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]
class airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2StopApplicationOperator(application_name, force=False, wait_for_completion=True, waiter_delay=60, waiter_max_attempts=20, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.kinesis_analytics.KinesisAnalyticsV2Hook]

Stop an AWS Managed Service for Apache Flink application.

See also

For more information on how to use this operator, take a look at the guide: Stop an Amazon Managed Service for Apache Flink Application

Parameters

  • application_name (str) – The name of your application. (templated)

  • force (bool) – Set to true to force the application to stop. If you set Force to true, Managed Service for Apache Flink stops the application without taking a snapshot. (templated)

  • wait_for_completion (bool) – Whether to wait for job to stop. (default: True)

  • waiter_delay (int) – Time in seconds to wait between status checks. (default: 60)

  • waiter_max_attempts (int) – Maximum number of attempts to check for job completion. (default: 20)

  • deferrable (bool) – If True, the operator will wait asynchronously for the job to stop. This implies waiting for completion. This mode requires aiobotocore module to be installed. (default: False)

  • aws_conn_id – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • region_name – AWS region_name. If not specified then the default boto3 behaviour is used.

  • verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

aws_hook_class[source]
ui_color = '#44b5e2'[source]
template_fields: collections.abc.Sequence[str][source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]

Was this entry helpful?