apache-airflow-providers-docker

Changelog¶

4.0.0¶

Note

This release of provider is only available for Airflow 2.9+ as explained in the Apache Airflow providers support policy.

Breaking changes¶

Warning

All deprecated classes, parameters and features have been removed from the Kubernetes provider package. The following breaking changes were introduced:

Decorators
- Deprecated parameter use_dill was removed. Please use serializer='dill' instead.
Operators
- Deprecated parameter use_dill was removed. Please use serializer='dill' instead.
- Deprecated parameter skip_exit_code was removed. Please use skip_on_exit_code instead.
- Deprecated method get_hook() was removed. Please use hook property instead.

Remove Provider Deprecations in Docker (#44583)

Misc¶

Bump minimum Airflow version in providers to Airflow 2.9.0 (#44956)
fix docker documentation auth url (#44112)
Update DAG example links in multiple providers documents (#44034)

3.14.1¶

Bug Fixes¶

Fix logs with leading spaces in the Docker operator (#33692) (#43840)

Misc¶

Move python operator to Standard provider (#42081)

3.14.0¶

Features¶

Add logging device and logging device options to DockerSwarmOperator (#41416)
Add retrieve output docker swarm operator (#41531)

3.13.0¶

Note

This release of provider is only available for Airflow 2.8+ as explained in the Apache Airflow providers support policy.

Misc¶

feat(docker): Replace 'use_dill' with 'serializer' (#41356)
Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)

3.12.3¶

Bug Fixes¶

DockerSwarmOperator: Support line breaks in service logs (#40705)

3.12.2¶

Bug Fixes¶

DockerOperator TaskFlow - correct argyments in python command (#39620)

Misc¶

Improve logging behavior of DockerOperator (#40489)

3.12.1¶

Misc¶

Bump minimum docker version to 7.1.0 (#39839)

3.12.0¶

Features¶

Add args to docker service ContainerSpec (#39464)
Add support to define Resources on DockerSwarmOperator (#39027)

Misc¶

Faster 'airflow_version' imports (#39552)
Simplify 'airflow_version' imports (#39497)
Limit requests in botocore upgrade test (#39747)
Pin requests due to incompatibility with docker-py (#39740)

3.11.0¶

Note

This release of provider is only available for Airflow 2.7+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump minimum Airflow version in providers to Airflow 2.7.0 (#39240)

3.10.0¶

Note

The standard DOCKER_HOST environment variable now overrides the default value of the docker_url parameter when set. If DOCKER_HOST is set but you want to use the previous default value, then you must explicitly set docker_url="unix://var/run/docker.sock" in the DockerOperator constructor or @task.docker decorator.

Features¶

Improve 'DockerOperator' to support multiple Docker hosts (#38466)

Bug Fixes¶

Fix deprecated 'DockerOperator' operator arguments in 'MappedOperator' (#38379)

Misc¶

Remove redundant compatibility usage of importlib_metadata (#38368)
DockerOperator: use DOCKER_HOST as default for docker_url (#38387)

3.9.2¶

Bug Fixes¶

Fix construct 'docker.TLSConfig' for 'docker>=7' (#37481)

3.9.1¶

Bug Fixes¶

Allow DockerOperator.skip_on_exit_code to be zero (#36360)

Misc¶

Remove remaining Airflow 2.5 backcompat code from Docker Provider (#36325)

3.9.0¶

Note

This release of provider is only available for Airflow 2.6+ as explained in the Apache Airflow providers support policy.

Bug Fixes¶

Fix 'enable_logging=True' not working in 'DockerSwarmOperator' (#35677)
Fix broken log streaming from #35677 (#36127)

Misc¶

Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017)
Follow BaseHook connection fields method signature in child classes (#36086)

3.8.2¶

Misc¶

Refactor docker operator attribute validations and docs (#35571)

3.8.1¶

Bug Fixes¶

fix '_DockerDecoratedOperator' module type attribute pickle error (#35293)

3.8.0¶

Note

This release of provider is only available for Airflow 2.5+ as explained in the Apache Airflow providers support policy.

Features¶

Add ulimits parameter to DockerOperator (#34284)

Misc¶

Bump min airflow version of providers (#34728)
Deprecate get_hook method in DockerOperator (#34432)

3.7.5¶

Misc¶

Cleanup Docker operator logging (#33914)
Replace sequence concatenation by unpacking in Airflow providers (#33933)
Use literal dict instead of calling dict() in providers (#33761)
Replace type func by isinstance in DockerOperator (#33759)

3.7.4¶

Misc¶

Refactor: Improve detection of duplicates and list sorting (#33675)
Simplify conditions on len() in other providers (#33569)
Replace repr() with proper formatting (#33520)

3.7.3¶

Misc¶

Refactor: Simplify code in providers/docker (#33232)

3.7.2¶

Misc¶

Get rid of Python2 numeric relics (#33050)

3.7.1¶

Note

This release dropped support for Python 3.7

Misc¶

Remove Python 3.7 support (#30963)

3.7.0¶

Note

This release of provider is only available for Airflow 2.4+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump minimum Airflow version in providers (#30917)

3.6.0¶

Features¶

Add multiple exit code handling in skip logic for 'DockerOperator' and 'KubernetesPodOperator' (#30769)
In 'DockerOperator', adding an attribute 'tls_verify' to choose whether to validate certificate (#30309) (#30310)

Misc¶

Deprecate 'skip_exit_code' in 'DockerOperator' and 'KubernetesPodOperator' (#30733)

3.5.1¶

Bug Fixes¶

fix template_fields in the decorator 'task.docker' (#29586)

3.5.0¶

Features¶

Add correct widgets in Docker Hook (#28700)
Make docker operators always use 'DockerHook' for API calls (#28363)
Skip DockerOperator task when it returns a provided exit code (#28996)

Bug Fixes¶

Fix label name for 'reauth' field in Docker Connection (#28974)

3.4.0¶

Features¶

add hostname argument to DockerOperator (#27822)
Move min airflow version down for Docker Provider to 2.3.0 (#28648)

3.3.0¶

Note

This release of provider is only available for Airflow 2.3+ as explained in the Apache Airflow providers support policy.

Misc¶

Move min airflow version to 2.3.0 for all providers (#27196)

Features¶

Add ipc_mode for DockerOperator (#27553)
Add env-file parameter to Docker Operator (#26951)

3.2.0¶

Features¶

Add logging options to docker operator (#26653)
Add pre-commit hook for custom_operator_name (#25786)
Implement ExternalPythonOperator (#25780)

Bug Fixes¶

3.1.0¶

Features¶

Force-remove container after DockerOperator execution (#23160)

Bug Fixes¶

'DockerOperator' fix cli.logs giving character array instead of string (#24726)

3.0.0¶

Breaking changes¶

Note

This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow providers support policy.

Misc¶

Remove 'xcom_push' from 'DockerOperator' (#23981)
docker new system test (#23167)

2.7.0¶

Features¶

Add 'device_requests' parameter to 'DockerOperator' (#23554)

Bug Fixes¶

Fix new MyPy errors in main (#22884)

2.6.0¶

Features¶

Add timeout parameter to 'DockerOperator' (#22502)

2.5.2¶

Bug Fixes¶

Fix mistakenly added install_requires for all providers (#22382)

Misc¶

Correct 'multiple_outputs' param descriptions mentioning lists/tuples (#22371)

2.5.1¶

Bug Fixes¶

Avoid trying to kill container when it did not succeed for Docker (#22145)

Misc¶

Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)

2.5.0¶

Features¶

added docker network_mode options (#21986)

Misc¶

Support for Python 3.10

2.4.1¶

Bug Fixes¶

Fixes Docker xcom functionality (#21175)
Fix docker behaviour with byte lines returned (#21429)

2.4.0¶

Features¶

Allow DockerOperator's image to be templated (#19997)

2.3.0¶

Features¶

Add support of placement in the DockerSwarmOperator (#18990)

Bug Fixes¶

Fixup string concatenations (#19099)
Remove the docker timeout workaround (#18872)

Other¶

Move docker decorator example dag to docker provider (#18739)

2.2.0¶

Features¶

Add a Docker Taskflow decorator (#15330)

This version of Docker Provider has a new feature - TaskFlow decorator that only works in Airflow 2.2. If you try to use the decorator in pre-Airflow 2.2 version you will get an error:

AttributeError: '_TaskDecorator' object has no attribute 'docker'

2.1.1¶

Features¶

Add support for configs, secrets, networks and replicas for DockerSwarmOperator (#17474)

Misc¶

Optimise connection importing for Airflow 2.2.0

2.1.0¶

Features¶

Adds option to disable mounting temporary folder in DockerOperator (#16932)

Bug Fixes¶

[FIX] Docker provider - retry docker in docker (#17061)
fix string encoding when using xcom / json (#13536)
if xcom_all is set to False, only the last line of the log (separated by \n) will be included in the XCom value

The DockerOperator in version 2.0.0 did not work for remote Docker Engine or Docker-In-Docker case. That was an unintended side effect of #15843 that has been fixed in #16932. There is a fallback mode which will make Docker Operator works with warning and you will be able to remove the warning by using the new parameter to disable mounting the folder.

2.0.0¶

Breaking changes¶

Auto-apply apply_default decorator (#15667)

Warning

Due to apply_default decorator removal, this version of the provider requires Airflow 2.1.0+. If your Airflow version is < 2.1.0, and you want to install this provider version, first upgrade Airflow to at least version 2.1.0. Otherwise your Airflow package version will be upgraded automatically and you will have to manually run airflow upgrade db to complete the migration.

Replace DockerOperator's 'volumes' arg for 'mounts' (#15843)

The volumes parameter in airflow.providers.docker.operators.docker.DockerOperator and airflow.providers.docker.operators.docker_swarm.DockerSwarmOperator was replaced by the mounts parameter, which uses the newer mount syntax instead of --bind.