airflow.providers.amazon.aws.operators.quicksight

Classes

QuickSightCreateIngestionOperator

Creates and starts a new SPICE ingestion for a dataset; also helps to Refresh existing SPICE datasets.

Module Contents

class airflow.providers.amazon.aws.operators.quicksight.QuickSightCreateIngestionOperator(data_set_id, ingestion_id, ingestion_type='FULL_REFRESH', wait_for_completion=True, waiter_delay=30, waiter_max_attempts=4320, check_interval=None, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.quicksight.QuickSightHook]

Creates and starts a new SPICE ingestion for a dataset; also helps to Refresh existing SPICE datasets.

See also

For more information on how to use this operator, take a look at the guide: Amazon QuickSight create ingestion

Parameters:

  • data_set_id (str) – ID of the dataset used in the ingestion.

  • ingestion_id (str) – ID for the ingestion.

  • ingestion_type (str) – Type of ingestion. Values Can be INCREMENTAL_REFRESH or FULL_REFRESH. Default FULL_REFRESH.

  • wait_for_completion (bool) – If True, wait for the ingestion to reach a terminal state. (default: True)

  • waiter_delay (int) – Time in seconds to wait between status checks. (default: 30)

  • waiter_max_attempts (int) – Maximum number of attempts to check for completion. (default: 4320)

  • check_interval (int | None) – Deprecated, use waiter_delay instead.

  • deferrable (bool) – If True, the operator will wait asynchronously for the ingestion to complete. This implies waiting for completion. This mode requires aiobotocore module to be installed. (default: False, but can be overridden in config file by setting default_deferrable to True)

  • aws_conn_id – The Airflow connection used for AWS credentials. If this is None or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).

  • region_name – AWS region_name. If not specified then the default boto3 behaviour is used.

  • verify – Whether or not to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html

  • botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

aws_hook_class[source]
template_fields: collections.abc.Sequence[str][source]
ui_color = '#ffd700'[source]
data_set_id[source]
ingestion_id[source]
ingestion_type = 'FULL_REFRESH'[source]
wait_for_completion = True[source]
waiter_delay = 30[source]
waiter_max_attempts = 4320[source]
check_interval = None[source]
deferrable[source]
execute(context)[source]

Derive when creating an operator.

The main method to execute the task. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None, ingestion=None)[source]

Was this entry helpful?