tests.system.google.cloud.dataplex.example_dataplex¶

Example Airflow DAG that shows how to use Dataplex.

Attributes¶

`ENV_ID`
`PROJECT_ID`
`DAG_ID`
`BUCKET_NAME`
`SPARK_FILE_NAME`
`RESOURCE_DATA_BUCKET`
`LAKE_ID`
`REGION`
`SERVICE_ACC`
`SPARK_FILE_FULL_PATH`
`DATAPLEX_TASK_ID`
`TRIGGER_SPEC_TYPE`
`EXAMPLE_TASK_BODY`
`EXAMPLE_LAKE_BODY`
`create_bucket`
`test_run`

Module Contents¶

tests.system.google.cloud.dataplex.example_dataplex.ENV_ID[source]¶

tests.system.google.cloud.dataplex.example_dataplex.PROJECT_ID[source]¶

tests.system.google.cloud.dataplex.example_dataplex.DAG_ID = 'dataplex'[source]¶

tests.system.google.cloud.dataplex.example_dataplex.BUCKET_NAME = ''[source]¶

tests.system.google.cloud.dataplex.example_dataplex.SPARK_FILE_NAME = 'spark_example_pi.py'[source]¶

tests.system.google.cloud.dataplex.example_dataplex.RESOURCE_DATA_BUCKET = 'airflow-system-tests-resources'[source]¶

tests.system.google.cloud.dataplex.example_dataplex.LAKE_ID = ''[source]¶

tests.system.google.cloud.dataplex.example_dataplex.REGION = 'us-west1'[source]¶

tests.system.google.cloud.dataplex.example_dataplex.SERVICE_ACC = 'Uninferable@appspot.gserviceaccount.com'[source]¶

tests.system.google.cloud.dataplex.example_dataplex.SPARK_FILE_FULL_PATH = 'gs:///spark_example_pi.py'[source]¶

tests.system.google.cloud.dataplex.example_dataplex.DATAPLEX_TASK_ID = ''[source]¶

tests.system.google.cloud.dataplex.example_dataplex.TRIGGER_SPEC_TYPE = 'ON_DEMAND'[source]¶

tests.system.google.cloud.dataplex.example_dataplex.EXAMPLE_TASK_BODY[source]¶

tests.system.google.cloud.dataplex.example_dataplex.EXAMPLE_LAKE_BODY[source]¶

tests.system.google.cloud.dataplex.example_dataplex.create_bucket[source]¶

tests.system.google.cloud.dataplex.example_dataplex.test_run[source]¶

Was this entry helpful?