apache-airflow-providers-celery

Changelog

3.9.0

Note

This release of provider is only available for Airflow 2.9+ as explained in the Apache Airflow providers support policy.

Misc

  • Bump minimum Airflow version in providers to Airflow 2.9.0 (#44956)

  • AIP-81 Move CLI Commands to directories according to Hybrid, Local and Remote (#44538)

  • Remove AIP-44 configuration from the code (#44454)

3.8.5

Bug Fixes

  • Re-queue tassk when they are stuck in queued (#43520)

3.8.4

Misc

  • AIP-72: Remove DAG pickling (#43667)

  • Move python operator to Standard provider (#42081)

3.8.3

Bug Fixes

  • All executors should inherit from BaseExecutor (#41904)

  • Remove state sync during celery task processing (#41870)

Misc

  • Change imports to use Standard provider for BashOperator (#42252)

3.8.2

Misc

  • remove deprecated soft_fail from providers (#41710)

3.8.1

Bug Fixes

  • fix: Missing 'slots_occupied' in 'CeleryKubernetesExecutor' and 'LocalKubernetesExecutor' (#41602)

3.8.0

Note

This release of provider is only available for Airflow 2.8+ as explained in the Apache Airflow providers support policy.

Misc

  • Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)

  • Remove deprecated SubDags (#41390)

3.7.3

Bug Fixes

  • Increase broker's visibility timeout to 24hrs (#40879)

3.7.2

Bug Fixes

  • Fixing exception types to include TypeError, which is what is raised in (#40012)

  • catch sentry flush if exception happens in _execute_in_fork finally block (#40060)

Misc

  • Add PID and return code to _execute_in_fork logging (#40058)

3.7.1

Misc

  • Faster 'airflow_version' imports (#39552)

  • Simplify 'airflow_version' imports (#39497)

  • ECS Executor: Set tasks to RUNNING state once active (#39212)

  • Remove compat code for 2.7.0 - its now the min Airflow version (#39591)

  • misc: add comment about remove unused code (#39748)

3.7.0

Note

This release of provider is only available for Airflow 2.7+ as explained in the Apache Airflow providers support policy.

Misc

  • Bump minimum Airflow version in providers to Airflow 2.7.0 (#39240)

3.6.2

Bug Fixes

  • Ensure __exit__ is called in decorator context managers (#38383)

  • Don't dispose sqlalchemy engine when using internal api (#38562)

  • Use celery worker CLI from Airflow package for Airflow < 2.8.0 (#38879)

Misc

  • Allow to use 'redis'>=5 (#38385)

  • Reraise of AirflowOptionalProviderFeatureException should be direct (#38555)

3.6.1

Bug Fixes

  • Remove pid arg from celery option to fix duplicate pid issue, Move celery command to provider package (#36794)

  • Change AirflowTaskTimeout to inherit BaseException (#35653)

Misc

  • Migrate executor docs to respective providers (#37728)

3.6.0

Features

  • Add 'task_acks_late' configuration to Celery Executor (#37066)

Misc

  • improve info for prevent celery command autoscale misconfig (#36576)

3.5.2

Bug Fixes

  • Fix stacklevel in warnings.warn into the providers (#36831)

3.5.1

Bug Fixes

  • Fix 'sentinel_kwargs' load from ENV (#36318)

3.5.0

Note

This release of provider is only available for Airflow 2.6+ as explained in the Apache Airflow providers support policy.

Misc

  • Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017)

3.4.1

Bug Fixes

  • Fix _SECRET and _CMD broker configuration (#34782)

  • Remove sensitive information from Celery executor warning (#34954)

3.4.0

Note

This release of provider is only available for Airflow 2.5+ as explained in the Apache Airflow providers support policy.

Misc

  • Bump min airflow version of providers (#34728)

  • respect soft_fail argument when exception is raised for celery sensors (#34474)

3.3.4

Bug Fixes

  • Fix condition of update_task_state in celery executor (#34192)

Misc

  • Combine similar if logics in providers (#33987)

  • Limit celery by excluding 5.3.2 and 5.3.3 (#34031)

  • Replace try - except pass by contextlib.suppress in providers (#33980)

  • Improve modules import in Airflow providers by some of them into a type-checking block (#33754)

3.3.3

Bug Fixes

  • Fix dependencies for celery and opentelemetry for Python 3.8 (#33579)

Misc

  • Make auth managers provide their own airflow CLI commands (#33481)

  • Refactor Sqlalchemy queries to 2.0 style (Part 7) (#32883)

3.3.2

Misc

  • Add missing re2 dependency to cncf.kubernetes and celery providers (#33237)

  • Replace State by TaskInstanceState in Airflow executors (#32627)

3.3.1

Misc

  • aDd documentation generation for CLI commands from executors (#33081)

  • Get rid of Python2 numeric relics (#33050)

3.3.0

Note

This provider release is the first release that has Celery Executor and Celery Kubernetes Executor moved from the core apache-airflow package to a Celery provider package. It also expects apache-airflow-providers-cncf-kubernetes in version 7.4.0+ installed in order to use CeleryKubernetesExecutor. You can install the provider with cncf.kubernetes extra with pip install apache-airflow-providers-celery[cncf.kubernetes] to get the right version of the cncf.kubernetes provider installed.

Features

  • Move CeleryExecutor to the celery provider (#32526)

  • Add pre-Airflow-2-7 hardcoded defaults for config for older providers  (#32775)

  • [AIP-51] Executors vending CLI commands (#29055)

Misc

  • Move all k8S classes to cncf.kubernetes provider (#32767)

  • Add Executors discovery and documentation (#32532)

  • Move default_celery.py to inside the provider (#32628)

  • Raise original import error in CLI vending of executors (#32931)

3.2.1

Note

This release dropped support for Python 3.7

Misc

  • Add note about dropping Python 3.7 for providers (#32015)

3.2.0

Note

This release of provider is only available for Airflow 2.4+ as explained in the Apache Airflow providers support policy.

Misc

  • Bump minimum Airflow version in providers (#30917)

3.1.0

Note

This release of provider is only available for Airflow 2.3+ as explained in the Apache Airflow providers support policy.

Misc

  • Move min airflow version to 2.3.0 for all providers (#27196)

3.0.0

Breaking changes

Note

This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow providers support policy.

2.1.4

Misc

  • Update our approach for executor-bound dependencies (#22573)

2.1.3

Bug Fixes

  • Fix mistakenly added install_requires for all providers (#22382)

2.1.2

Misc

  • Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)

2.1.1

Misc

  • Support for Python 3.10

2.1.0

Features

  • The celery provider is converted to work with Celery 5 following airflow 2.2.0 change of Celery version

2.0.0

Breaking changes

  • Auto-apply apply_default decorator (#15667)

Warning

Due to apply_default decorator removal, this version of the provider requires Airflow 2.1.0+. If your Airflow version is < 2.1.0, and you want to install this provider version, first upgrade Airflow to at least version 2.1.0. Otherwise your Airflow package version will be upgraded automatically and you will have to manually run airflow upgrade db to complete the migration.

1.0.1

Updated documentation and readme files.

1.0.0

Initial version of the provider.

Was this entry helpful?