apache-airflow-providers-apache-livy

Changelog¶

4.4.3¶

Release Date: 2025-09-22

Bug Fixes¶

Fix: use merged headers in LivyAsyncHook API call (#55072)

4.4.2¶

Release Date: 2025-08-02

Misc¶

Add Python 3.13 support for Airflow. (#46891)
Remove type ignore across codebase after mypy upgrade (#53243)
Remove upper-binding for "python-requires" (#52980)
Temporarily switch to use >=,< pattern instead of '~=' (#52967)

4.4.1¶

Release Date: 2025-07-06

Misc¶

Move 'BaseHook' implementation to task SDK (#51873)
Disable UP038 ruff rule and revert mandatory 'X | Y' in insintance checks (#52644)
Replace models.BaseOperator to Task SDK for apache/livy (#52499)
Drop support for Python 3.9 (#52072)
Use BaseSensorOperator from task sdk in providers (#52296)

4.4.0¶

Release Date: 2025-05-18

Note

This release of provider is only available for Airflow 2.10+ as explained in the Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>_.

Features¶

add root parent information to OpenLineage events (#49237)

Misc¶

Bump min Airflow version in providers to 2.10 (#49843)

4.3.2¶

Release Date: 2025-04-19

Misc¶

remove superfluous else block (#49199)

4.3.1¶

Release Date: 2025-04-14

Misc¶

Remove sqlite provider from preinstalled and use aiosqlite as core dep (#48966)

4.3.0¶

Release Date: 2025-04-09

Features¶

add OpenLineage configuration injection to LivyOperator (#47564)

4.2.1¶

Release Date: 2025-03-13

Misc¶

Upgrade flit to 3.11.0 (#46938)

4.2.0¶

Release Date: 2025-02-26

Note

This version has no code changes. It’s released due to yank of previous version due to packaging issues.

4.1.0¶

Release Date: 2025-02-21

Features¶

Add 'endpoint_prefix' to 'LivyHook' (#45811)

Misc¶

Satisfy Mypy for Livy Hook (#46157)

4.0.0¶

Release Date: 2024-12-26

Note

This release of provider is only available for Airflow 2.9+ as explained in the Apache Airflow providers support policy.

Breaking changes¶

Warning

All deprecated classes, parameters and features have been removed from the Livy provider package. The following breaking changes were introduced:

Operators
- Removed get_hook method from LivyOperator. Use hook property instead

Remove Provider Deprecations in Apache Livy (#44631)

Misc¶

Bump minimum Airflow version in providers to Airflow 2.9.0 (#44956)
Update DAG example links in multiple providers documents (#44034)

3.9.2¶

Release Date: 2024-10-31

Misc¶

Set lower bound to asgiref>=2.3.0 (#43001)

3.9.1¶

Release Date: 2024-09-24

Bug Fixes¶

bugfix/livy-set-base-url (#41658)

3.9.0¶

Release Date: 2024-08-22

Note

This release of provider is only available for Airflow 2.8+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)

3.8.1¶

Release Date: 2024-05-30

Misc¶

Faster 'airflow_version' imports (#39552)
Simplify 'airflow_version' imports (#39497)

3.8.0¶

Release Date: 2024-05-06

Note

This release of provider is only available for Airflow 2.7+ as explained in the Apache Airflow providers support policy.

Bug Fixes¶

Fix bug in LivyOperator when its trigger times out (#38916)
Fix 'polling_interval' parameter docs in LivyOperator (#38979)

Misc¶

Bump minimum Airflow version in providers to Airflow 2.7.0 (#39240)
Update docstring 'LivyOperator' retry_args and deferrable docs (#39266)

3.7.3¶

Release Date: 2024-03-08

Bug Fixes¶

Fix rendering 'LivyOperator.spark_params' (#37361)

3.7.2¶

Release Date: 2024-02-15

Misc¶

Bump aiohttp min version to avoid CVE-2024-23829 and CVE-2024-23334 (#37110)

3.7.1¶

Release Date: 2024-01-10

Bug Fixes¶

Fix assignment of template field in '__init__' in 'livy.py' (#36490)

3.7.0¶

Release Date: 2023-12-12

Note

This release of provider is only available for Airflow 2.6+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017)

3.6.0¶

Release Date: 2023-10-17

Note

This release of provider is only available for Airflow 2.5+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump min airflow version of providers (#34728)
Consolidate hook management in LivyOperator (#34431)

3.5.4¶

Release Date: 2023-09-12

Misc¶

Refactor regex in providers (#33898)
Improve modules import in Airflow providers by some of them into a type-checking block (#33754)

3.5.3¶

Release Date: 2023-08-29

Misc¶

Refactor: Remove useless str() calls (#33629)
Refactor: Simplify code in Apache/Alibaba providers (#33227)
Simplify conditions on len() in providers/apache (#33564)

3.5.2¶

Release Date: 2023-07-09

Bug Fixes¶

Fix 'LivyHook' TypeError exception on 'session_id' log format (#32051)

Misc¶

Add default_deferrable config (#31712)

3.5.1¶

Release Date: 2023-06-23

Note

This release dropped support for Python 3.7

Bug Fixes¶

Push Spark appId to XCOM for LivyOperator with deferrable mode (#31201)

Misc¶

Optimize deferred mode execution (#31685)

3.5.0¶

Release Date: 2023-05-22

Note

This release of provider is only available for Airflow 2.4+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump minimum Airflow version in providers (#30917)

3.4.0¶

Release Date: 2023-04-06

Features¶

Add non login-password auth support for SimpleHttpOpeator (#29206)

3.3.0¶

Release Date: 2023-03-06

Features¶

Add Livy Operator with deferrable mode (#29047)

3.2.0¶

Release Date: 2022-11-18

Note

This release of provider is only available for Airflow 2.3+ as explained in the Apache Airflow providers support policy.

Misc¶

Move min airflow version to 2.3.0 for all providers (#27196)

Features¶

Add template to livy operator documentation (#27404)
Add Spark's 'appId' to xcom output (#27376)
add template field renderer to livy operator (#27321)

3.1.0¶

Release Date: 2022-08-14

Features¶

Add auth_type to LivyHook (#25183)

3.0.0¶

Release Date: 2022-06-13

Breaking changes¶

Note

This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow providers support policy.

2.2.3¶

Release Date: 2022-04-11

Bug Fixes¶

Fix: Add extra headers to all livy API requests instead of only to post_batch (#22510)

2.2.2¶

Release Date: 2022-03-26

Bug Fixes¶

Fix mistakenly added install_requires for all providers (#22382)

2.2.1¶

Release Date: 2022-03-19

Misc¶

Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)

2.2.0¶

Release Date: 2022-03-10

Features¶

Added retries to LivyHook #19384 (#21550)

Misc¶

Support for Python 3.10
add how-to guide for livy operator (#21529)

2.1.0¶

Release Date: 2021-09-03

Features¶

Fetching and logging livy session logs for LivyOperrator (#17393)

Misc¶

Optimise connection importing for Airflow 2.2.0

2.0.0¶

Release Date: 2021-06-23

Breaking changes¶

Auto-apply apply_default decorator (#15667)

Warning

Due to apply_default decorator removal, this version of the provider requires Airflow 2.1.0+. If your Airflow version is < 2.1.0, and you want to install this provider version, first upgrade Airflow to at least version 2.1.0. Otherwise your Airflow package version will be upgraded automatically and you will have to manually run airflow upgrade db to complete the migration.

1.1.0¶

Release Date: 2021-04-13

Features¶

Extend HTTP extra_options to LivyHook and operator (#14816)

1.0.1¶

Release Date: 2021-02-08

Updated documentation and readme files.

1.0.0¶

Release Date: 2020-12-14

Initial version of the provider.