Amazon Elastic Compute Cloud (EC2)¶

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable computing capacity—literally, servers in Amazon’s data centers—that you use to build and host your software systems.

Prerequisite Tasks¶

To use these operators, you must do a few things:

Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'
Detailed information is available Installation of Airflow®
Setup Connection.

Operators¶

Start an Amazon EC2 instance¶

To start an Amazon EC2 instance you can use EC2StartInstanceOperator.

tests/system/amazon/aws/example_ec2.py

start_instance = EC2StartInstanceOperator(
    task_id="start_instance",
    instance_id=instance_id,
)

Stop an Amazon EC2 instance¶

To stop an Amazon EC2 instance you can use EC2StopInstanceOperator.

tests/system/amazon/aws/example_ec2.py

stop_instance = EC2StopInstanceOperator(
    task_id="stop_instance",
    instance_id=instance_id,
)

Create and start an Amazon EC2 instance¶

To create and start an Amazon EC2 instance you can use EC2CreateInstanceOperator.

tests/system/amazon/aws/example_ec2.py

create_instance = EC2CreateInstanceOperator(
    task_id="create_instance",
    image_id=image_id,
    max_count=1,
    min_count=1,
    config=config,
)

Terminate an Amazon EC2 instance¶

To terminate an Amazon EC2 instance you can use EC2TerminateInstanceOperator.

tests/system/amazon/aws/example_ec2.py

terminate_instance = EC2TerminateInstanceOperator(
    task_id="terminate_instance",
    instance_ids=instance_id,
    wait_for_completion=True,
)

Reboot an Amazon EC2 instance¶

To reboot an Amazon EC2 instance you can use EC2RebootInstanceOperator.

tests/system/amazon/aws/example_ec2.py

reboot_instance = EC2RebootInstanceOperator(
    task_id="reboot_instace",
    instance_ids=instance_id,
)

Hibernate an Amazon EC2 instance¶

To hibernate an Amazon EC2 instance you can use EC2HibernateInstanceOperator.

tests/system/amazon/aws/example_ec2.py

hibernate_instance = EC2HibernateInstanceOperator(
    task_id="hibernate_instace",
    instance_ids=instance_id,
)

Sensors¶

Wait on an Amazon EC2 instance state¶

To check the state of an Amazon EC2 instance and wait until it reaches the target state you can use EC2InstanceStateSensor.

tests/system/amazon/aws/example_ec2.py

await_instance = EC2InstanceStateSensor(
    task_id="await_instance",
    instance_id=instance_id,
    target_state="running",
)

Reference¶

Boto3 Library Documentation for EC2

Was this entry helpful?