tests.system.google.cloud.dataproc.example_dataproc_flink¶

Example Airflow DAG for DataprocSubmitJobOperator with hadoop job.

Attributes¶

`ENV_ID`
`DAG_ID`
`PROJECT_ID`
`BUCKET_NAME`
`CLUSTER_NAME_BASE`
`CLUSTER_NAME_FULL`
`CLUSTER_NAME`
`REGION`
`OUTPUT_FOLDER`
`OUTPUT_PATH`
`CLUSTER_CONFIG`
`FLINK_JOB`
`create_bucket`
`test_run`

Module Contents¶

tests.system.google.cloud.dataproc.example_dataproc_flink.ENV_ID[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.DAG_ID = 'dataproc_flink'[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.PROJECT_ID[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.BUCKET_NAME = 'bucket_dataproc_flink_Uninferable'[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.CLUSTER_NAME_BASE = ''[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.CLUSTER_NAME_FULL = ''[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.CLUSTER_NAME = ''[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.REGION = 'europe-west1'[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.OUTPUT_FOLDER = 'wordcount'[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.OUTPUT_PATH = 'gs://bucket_dataproc_flink_Uninferable/wordcount/'[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.CLUSTER_CONFIG[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.FLINK_JOB[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.create_bucket[source]¶

tests.system.google.cloud.dataproc.example_dataproc_flink.test_run[source]¶

Was this entry helpful?