Authoring and Scheduling¶
Here you can find detailed documentation about advanced authoring and scheduling airflow DAGs. It’s recommended that you first review the pages in core concepts
Authoring
- Plugins
- Deferrable Operators & Triggers
- DAG File Processing
- Serialization
- Connections & Hooks
- Dynamic Task Mapping
- Simple mapping
- Mapping with non-TaskFlow operators
- Assigning multiple parameters to a non-TaskFlow operator
- Mapping over a task group
- Filtering items from a mapped task
- Transforming expanding data
- Combining upstream data (aka “zipping”)
- Concatenating multiple upstreams
- What data types can be expanded?
- How do templated fields and mapped arguments interact?
- Placing limits on mapped tasks
- Automatically skipping zero-length maps
Scheduling
- Cron & Time Intervals
- Time Zones
- Assets
- Quickstart
- What is an “Asset”?
- What is valid URI?
- Extra information on asset
- How to use assets in your DAGs
- Multiple assets
- Attaching extra information to an emitting asset event
- Fetching information from previously emitted asset events
- Fetching information from a triggering asset event
- Manipulating queued asset events through REST API
- Advanced asset scheduling with conditional expressions
- Example Use
- Dynamic data events emitting and asset creation through AssetAlias
- Combining asset and time-based schedules
- Timetables