apache-airflow-providers-apache-livy

Changelog¶

4.0.0¶

Note

This release of provider is only available for Airflow 2.9+ as explained in the Apache Airflow providers support policy.

Breaking changes¶

Warning

All deprecated classes, parameters and features have been removed from the Livy provider package. The following breaking changes were introduced:

Operators
- Removed get_hook method from LivyOperator. Use hook property instead

Remove Provider Deprecations in Apache Livy (#44631)

Misc¶

Bump minimum Airflow version in providers to Airflow 2.9.0 (#44956)
Update DAG example links in multiple providers documents (#44034)

3.9.2¶

Misc¶

Set lower bound to asgiref>=2.3.0 (#43001)

3.9.1¶

Bug Fixes¶

bugfix/livy-set-base-url (#41658)

3.9.0¶

Note

This release of provider is only available for Airflow 2.8+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)

3.8.1¶

Misc¶

Faster 'airflow_version' imports (#39552)
Simplify 'airflow_version' imports (#39497)

3.8.0¶

Note

This release of provider is only available for Airflow 2.7+ as explained in the Apache Airflow providers support policy.

Bug Fixes¶

Fix bug in LivyOperator when its trigger times out (#38916)
Fix 'polling_interval' parameter docs in LivyOperator (#38979)

Misc¶

Bump minimum Airflow version in providers to Airflow 2.7.0 (#39240)
Update docstring 'LivyOperator' retry_args and deferrable docs (#39266)

3.7.3¶

Bug Fixes¶

Fix rendering 'LivyOperator.spark_params' (#37361)

3.7.2¶

Misc¶

Bump aiohttp min version to avoid CVE-2024-23829 and CVE-2024-23334 (#37110)

3.7.1¶

Bug Fixes¶

Fix assignment of template field in '__init__' in 'livy.py' (#36490)

3.7.0¶

Note

This release of provider is only available for Airflow 2.6+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017)

3.6.0¶

Note

This release of provider is only available for Airflow 2.5+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump min airflow version of providers (#34728)
Consolidate hook management in LivyOperator (#34431)

3.5.4¶

Misc¶

Refactor regex in providers (#33898)
Improve modules import in Airflow providers by some of them into a type-checking block (#33754)

3.5.3¶

Misc¶

Refactor: Remove useless str() calls (#33629)
Refactor: Simplify code in Apache/Alibaba providers (#33227)
Simplify conditions on len() in providers/apache (#33564)

3.5.2¶

Bug Fixes¶

Fix 'LivyHook' TypeError exception on 'session_id' log format (#32051)

Misc¶

Add default_deferrable config (#31712)

3.5.1¶

Note

This release dropped support for Python 3.7

Bug Fixes¶

Push Spark appId to XCOM for LivyOperator with deferrable mode (#31201)

Misc¶

Optimize deferred mode execution (#31685)

3.5.0¶

Note

This release of provider is only available for Airflow 2.4+ as explained in the Apache Airflow providers support policy.

Misc¶

Bump minimum Airflow version in providers (#30917)

3.4.0¶

Features¶

Add non login-password auth support for SimpleHttpOpeator (#29206)

3.3.0¶

Features¶

Add Livy Operator with deferrable mode (#29047)

3.2.0¶

Note

This release of provider is only available for Airflow 2.3+ as explained in the Apache Airflow providers support policy.

Misc¶

Move min airflow version to 2.3.0 for all providers (#27196)

Features¶

Add template to livy operator documentation (#27404)
Add Spark's 'appId' to xcom output (#27376)
add template field renderer to livy operator (#27321)

3.1.0¶

Features¶

Add auth_type to LivyHook (#25183)

3.0.0¶

Breaking changes¶

Note

This release of provider is only available for Airflow 2.2+ as explained in the Apache Airflow providers support policy.

2.2.3¶

Bug Fixes¶

Fix: Add extra headers to all livy API requests instead of only to post_batch (#22510)

2.2.2¶

Bug Fixes¶

Fix mistakenly added install_requires for all providers (#22382)

2.2.1¶

Misc¶

Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)

2.2.0¶

Features¶

Added retries to LivyHook #19384 (#21550)

Misc¶

Support for Python 3.10
add how-to guide for livy operator (#21529)

2.1.0¶

Features¶

Fetching and logging livy session logs for LivyOperrator (#17393)

Misc¶

Optimise connection importing for Airflow 2.2.0

2.0.0¶

Breaking changes¶

Auto-apply apply_default decorator (#15667)

Warning

Due to apply_default decorator removal, this version of the provider requires Airflow 2.1.0+. If your Airflow version is < 2.1.0, and you want to install this provider version, first upgrade Airflow to at least version 2.1.0. Otherwise your Airflow package version will be upgraded automatically and you will have to manually run airflow upgrade db to complete the migration.

1.1.0¶

Features¶

Extend HTTP extra_options to LivyHook and operator (#14816)

1.0.1¶

Updated documentation and readme files.

1.0.0¶

Initial version of the provider.