Google¶
Airflow has support for the Google service.
All hooks are based on airflow.providers.google.common.hooks.base_google.GoogleBaseHook. Some integration
also use airflow.providers.google.common.hooks.discovery_api.
See the Google Cloud connection type documentation to configure connections to Google services.
Google Cloud¶
Airflow has extensive support for the Google Cloud.
Note
You can learn how to use Google Cloud integrations by analyzing the source code of the Google Cloud example DAGs
Services¶
These integrations allow you to perform various operations within the Google Cloud.
Google AutoML¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google BigQuery¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google BigQuery Data Transfer Service¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Bigtable¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Campaign Manager¶
- Operators:
airflow.providers.google.marketing_platform.operators.campaign_manager.- Hooks:
airflow.providers.google.marketing_platform.hooks.campaign_manager.- Sensors:
airflow.providers.google.marketing_platform.sensors.campaign_manager.- Guides:
- Provider:
- Product documentation:
Google Cloud AlloyDB¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Build¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Common¶
- Operators:
- Provider:
- Product documentation:
Google Cloud Data Loss Prevention (DLP)¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Firestore¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Functions¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Key Management Service (KMS)¶
- Hooks:
- Provider:
- Product documentation:
Google Cloud Life Sciences¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Memorystore¶
- Operators:
- Hooks:
- Guides:
Google Cloud Memorystore Operators, Google Cloud Memorystore Memcached Operators.
- Provider:
- Product documentation:
Google Cloud Natural Language¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud OS Login¶
- Hooks:
- Provider:
- Product documentation:
Google Cloud Pub/Sub¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Cloud Secret Manager¶
- Hooks:
- Provider:
- Product documentation:
Google Cloud Spanner¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Speech-to-Text¶
- Operators:
- Hooks:
- Guides:
Google Cloud Speech to Text Operators, Google Cloud Speech Translate Operators.
- Provider:
- Product documentation:
Google Cloud SQL¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Stackdriver¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Storage (GCS)¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Cloud Storage Transfer Service¶
- Operators:
airflow.providers.google.cloud.operators.cloud_storage_transfer_service.- Hooks:
airflow.providers.google.cloud.hooks.cloud_storage_transfer_service.- Sensors:
airflow.providers.google.cloud.sensors.cloud_storage_transfer_service.- Guides:
- Provider:
- Product documentation:
Google Cloud Tasks¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Cloud Text-to-Speech¶
- Operators:
airflow.providers.google.cloud.operators.text_to_speech,airflow.providers.google.cloud.operators.translate_speech.- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Translation¶
- Operators:
airflow.providers.google.cloud.operators.translate,airflow.providers.google.cloud.operators.translate_speech.- Hooks:
- Guides:
Google Cloud Translate Operators, Google Cloud Speech Translate Operators.
- Provider:
- Product documentation:
Google Cloud Video Intelligence¶
- Operators:
airflow.providers.google.cloud.operators.video_intelligence.- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Vision¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Workflows¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Compute Engine¶
- Operators:
- Hooks:
airflow.providers.google.cloud.hooks.compute,airflow.providers.google.cloud.hooks.compute_ssh.- Guides:
Google Compute Engine Operators, Google Compute Engine SSH Operators.
- Provider:
- Product documentation:
Google Data Catalog¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Data Fusion¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Dataflow¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Dataplex¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Dataprep¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Dataproc¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Dataproc Metastore¶
- Operators:
airflow.providers.google.cloud.operators.dataproc_metastore.- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Datastore¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Deployment Manager¶
- Hooks:
- Provider:
- Product documentation:
Google Kubernetes Engine¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Looker¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Machine Learning Engine¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Vertex AI¶
- Operators:
airflow.providers.google.cloud.operators.vertex_ai.dataset,airflow.providers.google.cloud.operators.vertex_ai.custom_job,airflow.providers.google.cloud.operators.vertex_ai.auto_ml,airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job,airflow.providers.google.cloud.operators.vertex_ai.endpoint_service,airflow.providers.google.cloud.operators.vertex_ai.hyperparameter_tuning_job,airflow.providers.google.cloud.operators.vertex_ai.model_service,airflow.providers.google.cloud.operators.vertex_ai.pipeline_job,airflow.providers.google.cloud.operators.vertex_ai.generative_model,airflow.providers.google.cloud.operators.vertex_ai.feature_store.- Hooks:
airflow.providers.google.cloud.hooks.vertex_ai.dataset,airflow.providers.google.cloud.hooks.vertex_ai.custom_job,airflow.providers.google.cloud.hooks.vertex_ai.auto_ml,airflow.providers.google.cloud.hooks.vertex_ai.batch_prediction_job,airflow.providers.google.cloud.hooks.vertex_ai.endpoint_service,airflow.providers.google.cloud.hooks.vertex_ai.hyperparameter_tuning_job,airflow.providers.google.cloud.hooks.vertex_ai.model_service,airflow.providers.google.cloud.hooks.vertex_ai.pipeline_job,airflow.providers.google.cloud.hooks.vertex_ai.generative_model,airflow.providers.google.cloud.hooks.vertex_ai.prediction_service,airflow.providers.google.cloud.hooks.vertex_ai.feature_store.- Sensors:
airflow.providers.google.cloud.sensors.vertex_ai.feature_store.- Guides:
- Provider:
- Product documentation:
Transfers¶
These integrations allow you to copy data from/to Google Cloud.
Presto to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Trino to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Common SQL to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Google Cloud Storage (GCS) to Google Drive¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Drive to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Apache Cassandra to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Google Calendar to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Spreadsheet to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Cloud Storage (GCS) to SSH File Transfer Protocol (SFTP)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
PostgreSQL to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google BigQuery to Common SQL¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Google BigQuery to MySQL¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google BigQuery to Microsoft SQL Server (MSSQL)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
Google Cloud BigQuery Transfer Operator to Microsoft SQL Server
- Python API:
- Provider:
Google BigQuery to PostgreSQL¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
airflow.providers.google.cloud.transfers.bigquery_to_postgres- Provider:
Google Cloud Storage (GCS) to Google BigQuery¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Cloud Storage (GCS) to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Facebook Ads to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
airflow.providers.google.cloud.transfers.facebook_ads_to_gcs- Provider:
SSH File Transfer Protocol (SFTP) to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Microsoft Azure Data Lake Storage to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Google BigQuery to Google BigQuery¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
airflow.providers.google.cloud.transfers.bigquery_to_bigquery- Provider:
MySQL to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Oracle to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Cloud Storage (GCS) to Google Spreadsheet¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Local to Google Cloud Storage (GCS)¶
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google BigQuery to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
Google Cloud BigQuery Transfer Operator to Google Cloud Storage
- Python API:
- Provider:
Google Cloud Storage (GCS) to Local¶
- Source product documentation:
- Operator guide:
Downloads data from Google Cloud Storage to Local Filesystem
- Python API:
- Provider:
Salesforce to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Ads to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Microsoft SQL Server (MSSQL) to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Microsoft Azure Blob Storage to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
Azure Blob Storage to Google Cloud Storage (GCS) Transfer Operator
- Python API:
- Provider:
Google Cloud Storage (GCS) to Presto¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Cloud Storage (GCS) to Samba¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Cloud Storage (GCS) to Snowflake¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Google Cloud Storage (GCS) to Trino¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Cloud Storage (GCS) to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Glacier to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Marketing Platform¶
Note
You can learn how to use Google Marketing Platform integrations by analyzing the source code of the example DAGs.
Google Ads¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Analytics (GA4)¶
- Operators:
airflow.providers.google.marketing_platform.operators.analytics_admin.- Hooks:
airflow.providers.google.marketing_platform.hooks.analytics_admin.- Guides:
- Provider:
- Product documentation:
Google Display&Video 360¶
- Operators:
airflow.providers.google.marketing_platform.operators.display_video.- Hooks:
airflow.providers.google.marketing_platform.hooks.display_video.- Sensors:
airflow.providers.google.marketing_platform.sensors.display_video.- Guides:
- Provider:
- Product documentation:
Google Search Ads 360¶
- Operators:
airflow.providers.google.marketing_platform.operators.search_ads.- Hooks:
airflow.providers.google.marketing_platform.hooks.search_ads.- Guides:
- Provider:
- Product documentation:
Other Google operators and hooks¶
Google¶
- Hooks:
- Provider:
- Product documentation:
Google Calendar¶
- Hooks:
- Provider:
- Product documentation:
Google Cloud Batch¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Cloud Composer¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Cloud Dataform¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Google Cloud Run¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Discovery API¶
- Hooks:
- Provider:
- Product documentation:
Google Drive¶
- Hooks:
- Sensors:
- Provider:
- Product documentation:
Google LevelDB¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Google Spreadsheet¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation: