AWS: Amazon Web Services¶
Airflow has support for Amazon Web Services.
All hooks are based on airflow.providers.amazon.aws.hooks.base_aws.
Services¶
These integrations allow you to perform various operations within the Amazon Web Services.
Amazon Appflow¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Amazon Athena¶
- Operators:
- Hooks:
airflow.providers.amazon.aws.hooks.athena,airflow.providers.amazon.aws.hooks.athena_sql.- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon Bedrock¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon Chime¶
- Hooks:
- Provider:
- Product documentation:
Amazon CloudFormation¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon CloudWatch Logs¶
- Hooks:
- Provider:
- Product documentation:
Amazon Comprehend¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon DataSync¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Amazon DynamoDB¶
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon EC2¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon ECS¶
- Operators:
- Hooks:
- Sensors:
- Provider:
- Product documentation:
Amazon Elastic Container Registry (ECR)¶
- Hooks:
- Provider:
- Product documentation:
Amazon Elastic Kubernetes Service (EKS)¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon ElastiCache¶
- Hooks:
airflow.providers.amazon.aws.hooks.elasticache_replication_group.- Provider:
- Product documentation:
Amazon EMR¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon EMR on EKS¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon EventBridge¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Amazon Glacier¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon Kinesis Data Firehose¶
- Hooks:
- Provider:
- Product documentation:
Amazon Managed Service for Apache Flink¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon Neptune¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Amazon OpenSearch Serverless¶
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon QuickSight¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon RDS¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon Redshift¶
- Operators:
airflow.providers.amazon.aws.operators.redshift_cluster,airflow.providers.amazon.aws.operators.redshift_data.- Hooks:
airflow.providers.amazon.aws.hooks.redshift_sql,airflow.providers.amazon.aws.hooks.redshift_cluster,airflow.providers.amazon.aws.hooks.redshift_data.- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon SageMaker¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon SecretsManager¶
- Hooks:
- Provider:
- Product documentation:
Amazon Simple Email Service (SES)¶
- Hooks:
- Guides:
- Provider:
- Product documentation:
Amazon Simple Notification Service (SNS)¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
Amazon Simple Queue Service (SQS)¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon Simple Storage Service (S3)¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Amazon Systems Manager (SSM)¶
- Hooks:
- Provider:
- Product documentation:
Amazon Verified Permissions¶
- Hooks:
- Provider:
- Product documentation:
Amazon Web Services¶
- Operators:
- Hooks:
- Sensors:
- Provider:
- Product documentation:
AWS Batch¶
- Operators:
- Hooks:
airflow.providers.amazon.aws.hooks.batch_client,airflow.providers.amazon.aws.hooks.batch_waiters.- Sensors:
- Guides:
- Provider:
- Product documentation:
AWS Database Migration Service¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
AWS Glue¶
- Operators:
airflow.providers.amazon.aws.operators.glue,airflow.providers.amazon.aws.operators.glue_crawler.- Hooks:
airflow.providers.amazon.aws.hooks.glue,airflow.providers.amazon.aws.hooks.glue_crawler,airflow.providers.amazon.aws.hooks.glue_catalog.- Sensors:
airflow.providers.amazon.aws.sensors.glue,airflow.providers.amazon.aws.sensors.glue_crawler,airflow.providers.amazon.aws.sensors.glue_catalog_partition.- Guides:
- Provider:
- Product documentation:
AWS Glue DataBrew¶
- Operators:
- Hooks:
- Guides:
- Provider:
- Product documentation:
AWS Lambda¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
AWS Security Token Service (STS)¶
- Hooks:
- Provider:
- Product documentation:
AWS Step Functions¶
- Operators:
- Hooks:
- Sensors:
- Guides:
- Provider:
- Product documentation:
Transfers¶
These integrations allow you to copy data from/to Amazon Web Services.
Amazon Simple Storage Service (S3) to Apache Hive¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to MySQL¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to Snowflake¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to Teradata¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon DynamoDB to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google Cloud Storage (GCS) to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Glacier to Google Cloud Storage (GCS)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Google to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Apache Hive to Amazon DynamoDB¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Hypertext Transfer Protocol (HTTP) to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Internet Message Access Protocol (IMAP) to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
airflow.providers.amazon.aws.transfers.imap_attachment_to_s3- Provider:
MongoDB to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Redshift to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to Amazon Redshift¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to SSH File Transfer Protocol (SFTP)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
SSH File Transfer Protocol (SFTP) to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to File Transfer Protocol (FTP)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Exasol to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
File Transfer Protocol (FTP) to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Salesforce to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Local to Amazon Simple Storage Service (S3)¶
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Common SQL to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to Common SQL¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Web Services to Amazon Web Services¶
- Source product documentation:
- Target product documentation:
- Python API:
- Provider:
Microsoft Azure Blob Storage to Amazon Simple Storage Service (S3)¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to Amazon DynamoDB¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider:
Amazon Simple Storage Service (S3) to Microsoft Azure Blob Storage¶
- Source product documentation:
- Target product documentation:
- Operator guide:
- Python API:
- Provider: