airflow.example_dags.example_assets¶
Example DAG for demonstrating the behavior of the Assets feature in Airflow, including conditional and asset expression-based scheduling.
Notes on usage:
Turn on all the DAGs.
asset_produces_1 is scheduled to run daily. Once it completes, it triggers several DAGs due to its asset being updated. asset_consumes_1 is triggered immediately, as it depends solely on the asset produced by asset_produces_1. consume_1_or_2_with_asset_expressions will also be triggered, as its condition of either asset_produces_1 or asset_produces_2 being updated is satisfied with asset_produces_1.
asset_consumes_1_and_2 will not be triggered after asset_produces_1 runs because it requires the asset from asset_produces_2, which has no schedule and must be manually triggered.
After manually triggering asset_produces_2, several DAGs will be affected. asset_consumes_1_and_2 should run because both its asset dependencies are now met. consume_1_and_2_with_asset_expressions will be triggered, as it requires both asset_produces_1 and asset_produces_2 assets to be updated. consume_1_or_2_with_asset_expressions will be triggered again, since it’s conditionally set to run when either asset is updated.
consume_1_or_both_2_and_3_with_asset_expressions demonstrates complex asset dependency logic. This DAG triggers if asset_produces_1 is updated or if both asset_produces_2 and dag3_asset are updated. This example highlights the capability to combine updates from multiple assets with logical expressions for advanced scheduling.
conditional_asset_and_time_based_timetable illustrates the integration of time-based scheduling with asset dependencies. This DAG is configured to execute either when both asset_produces_1 and asset_produces_2 assets have been updated or according to a specific cron schedule, showcasing Airflow’s versatility in handling mixed triggers for asset and time-based scheduling.
The DAGs asset_consumes_1_never_scheduled and asset_consumes_unknown_never_scheduled will not run automatically as they depend on assets that do not get updated or are not produced by any scheduled tasks.