• Home
• Authoring and Scheduling

Authoring and Scheduling

Here you can find detailed documentation about advanced authoring and scheduling Airflow dags. It’s recommended that you first review the pages in core concepts

Authoring

• Deferrable Operators & Triggers
  • Using Deferrable Operators
  • Writing Deferrable Operators
  • High Availability
  • Difference between Mode=’reschedule’ and Deferrable=True in Sensors
• Serialization
  • Airflow Object
• Connections & Hooks
  • Hooks
  • Custom connections
• Dynamic Task Mapping
  • Simple mapping
  • Mapping with non-TaskFlow operators
  • Assigning multiple parameters to a non-TaskFlow operator
  • Mapping over a task group
  • Filtering items from a mapped task
  • Transforming expanding data
  • Combining upstream data (aka “zipping”)
  • Concatenating multiple upstreams
  • What data types can be expanded?
  • How do templated fields and mapped arguments interact?
  • Placing limits on mapped tasks
  • Automatically skipping zero-length maps
• Asset Definitions
  • What is an “Asset”?
  • What is valid URI?
  • Extra information on asset
  • Creating a task to emit asset events
  • Dependency between @asset, @task, and classic operators
  • Output to multiple assets in one task
  • Dynamic data events emitting and asset creation through AssetAlias

Scheduling

• Cron & Time Intervals
  • Cron Presets
• Time Zones
  • Web UI
  • Concepts
  • Time zone aware dags
• Asset-Aware Scheduling
  • Quickstart
  • Schedule dags with assets
  • Multiple assets
  • Fetching information from a triggering asset event
  • Manipulating queued asset events through REST API
  • Advanced asset scheduling with conditional expressions
  • Example Use
  • Combining asset and time-based schedules
• Timetables
  • Built-in Timetables
  • Timetables comparisons
• Event-Driven Scheduling
  • Supported triggers for event-driven scheduling
  • Use cases for event-driven dags

Previous Next

Was this entry helpful?