tests.system.google.cloud.bigtable.example_bigtable¶

Example Airflow DAG that creates and performs following operations on Cloud Bigtable: - creates an Instance - creates a Table - updates Cluster - waits for Table replication completeness - deletes the Table - deletes the Instance

This DAG relies on the following environment variables:

GCP_PROJECT_ID - Google Cloud project
CBT_INSTANCE_ID - desired ID of a Cloud Bigtable instance
CBT_INSTANCE_DISPLAY_NAME - desired human-readable display name of the Instance
CBT_INSTANCE_TYPE - type of the Instance, e.g. 1 for DEVELOPMENT
See https://googleapis.github.io/google-cloud-python/latest/bigtable/instance.html#google.cloud.bigtable.instance.Instance
CBT_INSTANCE_LABELS - labels to add for the Instance
CBT_CLUSTER_ID - desired ID of the main Cluster created for the Instance
CBT_CLUSTER_ZONE - zone in which main Cluster will be created. e.g. europe-west1-b
See available zones: https://cloud.google.com/bigtable/docs/locations
CBT_CLUSTER_NODES - initial amount of nodes of the Cluster
CBT_CLUSTER_NODES_UPDATED - amount of nodes for BigtableClusterUpdateOperator
CBT_CLUSTER_STORAGE_TYPE - storage for the Cluster, e.g. 1 for SSD
See https://googleapis.github.io/google-cloud-python/latest/bigtable/instance.html#google.cloud.bigtable.instance.Instance.cluster
CBT_TABLE_ID - desired ID of the Table
CBT_POKE_INTERVAL - number of seconds between every attempt of Sensor check

Module Contents¶

tests.system.google.cloud.bigtable.example_bigtable.ENV_ID[source]¶

tests.system.google.cloud.bigtable.example_bigtable.PROJECT_ID[source]¶

tests.system.google.cloud.bigtable.example_bigtable.DAG_ID = 'bigtable'[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_ID_BASE = ''[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_ID_1 = '-1'[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_ID_2 = '-2'[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_DISPLAY_NAME = 'instance-name'[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_DISPLAY_NAME_UPDATED = 'instance-name-updated'[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_TYPE[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_TYPE_PROD[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_LABELS: dict[str, str][source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_INSTANCE_LABELS_UPDATED[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_CLUSTER_ID = 'bigtable-cluster-id'[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_CLUSTER_ZONE = 'europe-west1-b'[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_CLUSTER_NODES = 3[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_CLUSTER_NODES_UPDATED = 5[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_CLUSTER_STORAGE_TYPE[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_TABLE_ID = 'bigtable-table-id'[source]¶

tests.system.google.cloud.bigtable.example_bigtable.CBT_POKE_INTERVAL = 60[source]¶

tests.system.google.cloud.bigtable.example_bigtable.create_instance_task[source]¶

tests.system.google.cloud.bigtable.example_bigtable.test_run[source]¶

`ENV_ID`
`PROJECT_ID`
`DAG_ID`
`CBT_INSTANCE_ID_BASE`
`CBT_INSTANCE_ID_1`
`CBT_INSTANCE_ID_2`
`CBT_INSTANCE_DISPLAY_NAME`
`CBT_INSTANCE_DISPLAY_NAME_UPDATED`
`CBT_INSTANCE_TYPE`
`CBT_INSTANCE_TYPE_PROD`
`CBT_INSTANCE_LABELS`
`CBT_INSTANCE_LABELS_UPDATED`
`CBT_CLUSTER_ID`
`CBT_CLUSTER_ZONE`
`CBT_CLUSTER_NODES`
`CBT_CLUSTER_NODES_UPDATED`
`CBT_CLUSTER_STORAGE_TYPE`
`CBT_TABLE_ID`
`CBT_POKE_INTERVAL`
`create_instance_task`
`test_run`

tests.system.google.cloud.bigtable.example_bigtable¶

Attributes¶

Module Contents¶