Apache Airflow Task SDK¶

The Apache Airflow Task SDK provides python-native interfaces for defining DAGs, executing tasks in isolated subprocesses and interacting with Airflow resources (e.g., Connections, Variables, XComs, Metrics, Logs, and OpenLineage events) at runtime. It also includes core execution-time components to manage communication between the worker and the Airflow scheduler/backend.

The goal of task-sdk is to decouple DAG authoring from Airflow internals (Scheduler, API Server, etc.), providing a forward-compatible, stable interface for writing and maintaining DAGs across Airflow versions. This approach reduces boilerplate and keeps your DAG definitions concise and readable.

1. Introduction and Getting Started¶

Below is a quick introduction and installation guide to get started with the Task SDK.

Installation¶

To install the Task SDK, run:

pip install apache-airflow-task-sdk

Getting Started¶

Define a basic DAG and task in just a few lines of Python:

/opt/airflow/airflow-core/src/airflow/example_dags/example_simplest_dag.py

from airflow.sdk import dag, task


@dag
def example_simplest_dag():
    @task
    def my_task():
        pass

    my_task()

2. Public Interface¶

Direct metadata database access from task code is now restricted. A dedicated Task Execution API handles all runtime interactions (state transitions, heartbeats, XComs, and resource fetching), ensuring isolation and security.

Airflow now supports a service-oriented architecture, enabling tasks to be executed remotely via a new Task Execution API. This API decouples task execution from the scheduler and introduces a stable contract for running tasks outside of Airflow’s traditional runtime environment.

To support remote execution, Airflow provides the Task SDK — a lightweight runtime environment for running Airflow tasks in external systems such as containers, edge environments, or other runtimes. This lays the groundwork for language-agnostic task execution and brings improved isolation, portability, and extensibility to Airflow-based workflows.

Airflow 3.0 also introduces a new airflow.sdk namespace that exposes the core authoring interfaces for defining DAGs and tasks. DAG authors should now import objects like airflow.sdk.DAG, airflow.sdk.dag(), and airflow.sdk.task() from airflow.sdk rather than internal modules. This new namespace provides a stable, forward-compatible interface for DAG authoring across future versions of Airflow.

3. DAG Authoring Enhancements¶

Writing your DAGs is now more consistent in Airflow 3.0. Use the stable airflow.sdk interface to define your workflows and tasks.

Why use `airflow.sdk`?¶

Decouple your DAG definitions from Airflow internals (Scheduler, API Server, etc.)
Enjoy a consistent API that won’t break across Airflow upgrades
Import only the classes and decorators you need, without installing the full Airflow core

Key imports from airflow.sdk

Classes

Decorators and helper functions

All DAGs must update their imports to refer to airflow.sdk instead of using internal Airflow modules directly. Deprecated legacy import paths, such as airflow.models.dag.DAG and airflow.decorator.task, will be removed in a future version of Airflow. Some utilities and helper functions currently used from airflow.utils.* and other modules will gradually be migrated to the Task SDK over the next minor releases. These upcoming updates aim to completely separate DAG creation from internal Airflow services. DAG authors can look forward to continuous improvements to airflow.sdk, with no backwards-incompatible changes to their existing code.

Legacy imports (deprecated):

# Airflow 2.x
from airflow.models import DAG
from airflow.decorators import task

Use instead:

# Airflow 3.x
from airflow.sdk import DAG, task

4. Example DAG References¶

Explore a variety of DAG examples and patterns in the Examples page.

5. Concepts¶

Discover the fundamental concepts that DAG authors need to understand when working with the Task SDK, including Airflow 2.x vs 3.x architectural differences, database access restrictions, and task lifecycle. For full details, see the Concepts page.

Airflow 2.x Architecture¶

Architectural Decoupling: Task Execution Interface (Airflow 3.x)¶

Airflow 3.x Task Execution API architecture diagram showing Execution API Server and worker subprocesses

6. API References¶

For the full public API reference, see the airflow.sdk API Reference page.