airflow.providers.amazon.aws.operators.dms

Module Contents

Classes

DmsCreateTaskOperator

Creates AWS DMS replication task.

DmsDeleteTaskOperator

Deletes AWS DMS replication task.

DmsDescribeTasksOperator

Describes AWS DMS replication tasks.

DmsStartTaskOperator

Starts AWS DMS replication task.

DmsStopTaskOperator

Stops AWS DMS replication task.

DmsDescribeReplicationConfigsOperator

Describes AWS DMS Serverless replication configurations.

DmsCreateReplicationConfigOperator

Creates an AWS DMS Serverless replication configuration.

DmsDeleteReplicationConfigOperator

Deletes an AWS DMS Serverless replication configuration.

DmsDescribeReplicationsOperator

Describes AWS DMS Serverless replications.

DmsStartReplicationOperator

Starts an AWS DMS Serverless replication.

DmsStopReplicationOperator

Stops an AWS DMS Serverless replication.
class airflow.providers.amazon.aws.operators.dms.DmsCreateTaskOperator(*, replication_task_id, source_endpoint_arn, target_endpoint_arn, replication_instance_arn, table_mappings, migration_type='full-load', create_task_kwargs=None, aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Creates AWS DMS replication task.

See also

For more information on how to use this operator, take a look at the guide: Create a replication task

Parameters

  • replication_task_id (str) – Replication task id

  • source_endpoint_arn (str) – Source endpoint ARN

  • target_endpoint_arn (str) – Target endpoint ARN

  • replication_instance_arn (str) – Replication instance ARN

  • table_mappings (dict) – Table mappings

  • migration_type (str) – Migration type (‘full-load’|’cdc’|’full-load-and-cdc’), full-load by default.

  • create_task_kwargs (dict | None) – Extra arguments for DMS replication task creation.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • region_name – AWS region_name. If not specified then the default boto3 behaviour is used.

  • verify – Whether or not to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
template_fields_renderers: ClassVar[dict][source]
execute(context)[source]

Create AWS DMS replication task from Airflow.

Returns

replication task arn

class airflow.providers.amazon.aws.operators.dms.DmsDeleteTaskOperator(*, replication_task_arn=None, **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Deletes AWS DMS replication task.

See also

For more information on how to use this operator, take a look at the guide: Delete a replication task

Parameters
aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
execute(context)[source]

Delete AWS DMS replication task from Airflow.

Returns

replication task arn

class airflow.providers.amazon.aws.operators.dms.DmsDescribeTasksOperator(*, describe_tasks_kwargs=None, **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Describes AWS DMS replication tasks.

See also

For more information on how to use this operator, take a look at the guide: Get details of replication tasks

Parameters
aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
template_fields_renderers: ClassVar[dict[str, str]][source]
execute(context)[source]

Describe AWS DMS replication tasks from Airflow.

Returns

Marker and list of replication tasks

Return type

tuple[str | None, list]

class airflow.providers.amazon.aws.operators.dms.DmsStartTaskOperator(*, replication_task_arn, start_replication_task_type='start-replication', start_task_kwargs=None, aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Starts AWS DMS replication task.

See also

For more information on how to use this operator, take a look at the guide: Start a replication task

Parameters

  • replication_task_arn (str) – Replication task ARN

  • start_replication_task_type (str) – Replication task start type (default=’start-replication’) (‘start-replication’|’resume-processing’|’reload-target’)

  • start_task_kwargs (dict | None) – Extra start replication task arguments

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • region_name – AWS region_name. If not specified then the default boto3 behaviour is used.

  • verify – Whether or not to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
template_fields_renderers[source]
execute(context)[source]

Start AWS DMS replication task from Airflow.

class airflow.providers.amazon.aws.operators.dms.DmsStopTaskOperator(*, replication_task_arn=None, **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Stops AWS DMS replication task.

See also

For more information on how to use this operator, take a look at the guide: Stop a replication task

Parameters
aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
execute(context)[source]

Stop AWS DMS replication task from Airflow.

class airflow.providers.amazon.aws.operators.dms.DmsDescribeReplicationConfigsOperator(*, filter=None, aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Describes AWS DMS Serverless replication configurations.

See also

For more information on how to use this operator, take a look at the guide: Describe a serverless replication config

Parameters

  • describe_config_filter – Filters block for filtering results.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be

aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
template_fields_renderers[source]
execute(context)[source]

Describe AWS DMS replication configurations.

Returns

List of replication configurations

Return type

list

class airflow.providers.amazon.aws.operators.dms.DmsCreateReplicationConfigOperator(*, replication_config_id, source_endpoint_arn, target_endpoint_arn, compute_config, replication_type, table_mappings, additional_config_kwargs=None, aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Creates an AWS DMS Serverless replication configuration.

See also

For more information on how to use this operator, take a look at the guide: Create a serverless replication config

Parameters

  • replication_config_id (str) – Unique identifier used to create a ReplicationConfigArn.

  • source_endpoint_arn (str) – ARN of the source endpoint

  • target_endpoint_arn (str) – ARN of the target endpoint

  • compute_config (dict[str, Any]) – Parameters for provisioning an DMS Serverless replication.

  • replication_type (str) – type of DMS Serverless replication

  • table_mappings (str) – JSON table mappings

  • tags – Key-value tag pairs

  • additional_config_kwargs (dict | None) – Additional configuration parameters for DMS Serverless replication. Passed directly to the API

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be

aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
template_fields_renderers[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.dms.DmsDeleteReplicationConfigOperator(*, replication_config_arn, wait_for_completion=True, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), waiter_delay=60, waiter_max_attempts=60, aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Deletes an AWS DMS Serverless replication configuration.

See also

For more information on how to use this operator, take a look at the guide: Delete a serverless replication configuration

Parameters

  • replication_config_arn (str) – ARN of the replication config

  • wait_for_completion (bool) – If True, waits for the replication config to be deleted before returning. If False, the operator will return immediately after the request is made.

  • deferrable (bool) – Run the operator in deferrable mode.

  • waiter_delay (int) – The number of seconds to wait between retries (default: 60).

  • waiter_max_attempts (int) – The maximum number of attempts to be made (default: 60).

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be

aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
VALID_STATES = ['failed', 'stopped', 'created'][source]
DELETING_STATES = ['deleting'][source]
TERMINAL_PROVISION_STATES = ['deprovisioned', ''][source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

handle_delete_wait()[source]
execute_complete(context, event=None)[source]
retry_execution(context, event=None)[source]
class airflow.providers.amazon.aws.operators.dms.DmsDescribeReplicationsOperator(*, filter=None, aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Describes AWS DMS Serverless replications.

See also

For more information on how to use this operator, take a look at the guide: Get the status of a serverless replication

Parameters

  • filter (list[dict[str, Any]] | None) – Filters block for filtering results.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be

aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
template_fields_renderer[source]
execute(context)[source]

Describe AWS DMS replications.

Returns

Replications

Return type

list[dict[str, Any]]

class airflow.providers.amazon.aws.operators.dms.DmsStartReplicationOperator(*, replication_config_arn, replication_start_type, cdc_start_time=None, cdc_start_pos=None, cdc_stop_pos=None, wait_for_completion=True, waiter_delay=30, waiter_max_attempts=60, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Starts an AWS DMS Serverless replication.

See also

For more information on how to use this operator, take a look at the guide: Start a serverless replication

Parameters

  • replication_config_arn (str) – ARN of the replication config

  • replication_start_type (str) – Type of replication.

  • cdc_start_time (datetime.datetime | str | None) – Start time of CDC

  • cdc_start_pos (str | None) – Indicates when to start CDC.

  • cdc_stop_pos (str | None) – Indicates when to stop CDC.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be

RUNNING_STATES = ['running'][source]
STARTABLE_STATES = ['stopped', 'failed', 'created'][source]
TERMINAL_STATES = ['failed', 'stopped', 'created'][source]
TERMINAL_PROVISION_STATES = ['deprovisioned', ''][source]
aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]
retry_execution(context, event=None)[source]
class airflow.providers.amazon.aws.operators.dms.DmsStopReplicationOperator(*, replication_config_arn, wait_for_completion=True, waiter_delay=30, waiter_max_attempts=60, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.dms.DmsHook]

Stops an AWS DMS Serverless replication.

See also

For more information on how to use this operator, take a look at the guide: Stop a serverless replication

Parameters

  • replication_config_arn (str) – ARN of the replication config

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be

STOPPED_STATES = ['stopped'][source]
NON_STOPPABLE_STATES = ['stopped'][source]
aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]

Was this entry helpful?