tests.system.google.cloud.dataflow.example_dataflow_yaml¶

Example Airflow DAG for Google Cloud Dataflow YAML service.

Requirements:: This test requires gcloud command (Google Cloud SDK) to be installed on the Airflow worker <https://cloud.google.com/sdk/docs/install>`__

Attributes¶

`PROJECT_ID`
`ENV_ID`
`DAG_ID`
`REGION`
`DATAFLOW_YAML_JOB_NAME`
`BQ_DATASET`
`BQ_INPUT_TABLE`
`BQ_OUTPUT_TABLE`
`DATAFLOW_YAML_PIPELINE_FILE_URL`
`SCHEMA`
`BQ_VARIABLES`
`BQ_VARIABLES_DEF`
`INSERT_ROWS_QUERY`
`create_bq_dataset`
`test_run`

tests.system.google.cloud.dataflow.example_dataflow_yaml.PROJECT_ID[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.DAG_ID = 'dataflow_yaml'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.REGION = 'europe-west2'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.DATAFLOW_YAML_JOB_NAME = ''[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.BQ_DATASET = ''[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.BQ_INPUT_TABLE = ''[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.BQ_OUTPUT_TABLE = ''[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.DATAFLOW_YAML_PIPELINE_FILE_URL = 'gs://airflow-system-tests-resources/dataflow/yaml/example_beam_yaml_bq.yaml'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.BQ_VARIABLES[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.BQ_VARIABLES_DEF[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.INSERT_ROWS_QUERY = "INSERT . VALUES ('John Doe', 900, 'USA'), ('Alice Storm', 1200, 'Australia'),('Bob Max', 1000,...[source]¶

tests.system.google.cloud.dataflow.example_dataflow_yaml.create_bq_dataset[source]¶