Google Data Studio (Looker) Operators¶

Data Studio (Looker) is a business intelligence software and big data analytics platform that helps you explore, analyze and share real-time business analytics easily.

Data Studio (Looker) has a Public API and associated SDK clients in different languages, which allow programmatic access to the Data Studio platform.

For more information visit Looker API documentation.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Install API libraries via pip.

pip install 'apache-airflow[google]'

Detailed information is available for Installation.

Setup a Data Studio connection in Airflow. You can check Managing Connections and Google Cloud Platform Data Studio (Looker) Connection

Start a PDT materialization job¶

To submit a PDT materialization job to Data Studio you need to provide a model and view name.

The job configuration can be submitted in synchronous (blocking) mode by using: LookerStartPdtBuildOperator.

The executable example below still imports the compatibility name LookerStartPdtBuildOperator. The preferred alias for new code is DataStudioStartPdtBuildOperator.

tests/system/google/cloud/looker/example_looker.py[source]

build_pdt_task = LookerStartPdtBuildOperator(
    task_id="build_pdt_task",
    looker_conn_id=LOOKER_CONNECTION_ID,
    model=LOOKER_MODEL,
    view=LOOKER_VIEW,
)

Alternatively, the job configuration can be submitted in asynchronous mode by using: LookerStartPdtBuildOperator and LookerCheckPdtBuildSensor.

The executable example below still imports the compatibility name LookerStartPdtBuildOperator. The preferred alias for new code is DataStudioStartPdtBuildOperator.

tests/system/google/cloud/looker/example_looker.py[source]

start_pdt_task_async = LookerStartPdtBuildOperator(
    task_id="start_pdt_task_async",
    looker_conn_id=LOOKER_CONNECTION_ID,
    model=LOOKER_MODEL,
    view=LOOKER_VIEW,
    asynchronous=True,
)

check_pdt_task_async_sensor = LookerCheckPdtBuildSensor(
    task_id="check_pdt_task_async_sensor",
    looker_conn_id=LOOKER_CONNECTION_ID,
    materialization_id=start_pdt_task_async.output,
    poke_interval=10,
)

There are more arguments to provide in the jobs than the examples show. For the complete list of arguments take a look at Data Studio operator arguments at airflow.providers.google.cloud.operators.looker.LookerStartPdtBuildOperator