Package apache-airflow-providers-openlineage¶

OpenLineage

This is detailed commit list of changes for versions provider package: openlineage. For high-level changelog, see package information including changelog.

2.0.0¶

Latest change: 2024-12-19

Commit	Committed	Subject
04ccef9b3a	2024-12-19	`feat: automatically inject OL info into spark job in DataprocSubmitJobOperator (#44477)`
2a33da0246	2024-12-18	`Remove references to AIRFLOW_V_2_9_PLUS (#44987)`
4b38bed76c	2024-12-16	`Bump min version of Providers to 2.9 (#44956)`
490b5e816b	2024-12-10	`Consistent way of checking Airflow version in providers (#44686)`
e26909df6a	2024-12-07	`add clear_number to OpenLineage's dagrun-level event run id generation (#44617)`
37236b6983	2024-12-06	`Remove Provider Deprecations in OpenLineage (#44636)`
7cc36104ab	2024-11-27	`add basic system tests for OpenLineage (#43643)`
05f935d2a6	2024-11-25	`utilize more information to deterministically generate OpenLineage run_id (#43936)`
1275fec92f	2024-11-24	`Use Python 3.9 as target version for Ruff & Black rules (#44298)`
a0f3353c47	2024-11-20	`Move Asset user facing components to task_sdk (#43773)`
123dadda0e	2024-11-15	`Rename execution_date to logical_date across codebase (#43902)`

1.14.0¶

Latest change: 2024-11-14

Commit	Committed	Subject
a53d9f6d25	2024-11-14	`Prepare docs for Nov 1st wave of providers (#44011)`
f60886cf36	2024-11-14	`add ProcessingEngineRunFacet to OL DAG Start event (#43213)`
e7194dff6a	2024-11-13	`Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916)`
c7c6547481	2024-11-06	`OpenLineage: accept whole config when instantiating OpenLineageClient. (#43740)`
80727d42ab	2024-11-06	`Temporarily limit openlineage to <1.24.0 (#43732)`
73f2eab680	2024-11-05	`serialize asset/dataset timetable conditions in OpenLineage info also for Airflow 2 (#43434)`
06088a3abc	2024-10-31	`Standard provider python operator (#42081)`

1.13.0¶

Latest change: 2024-10-27

Commit	Committed	Subject
78ff0a9970	2024-10-27	`Prepare docs for Oct 2nd wave of providers (#43409)`
0c4ed7a58e	2024-10-23	`Ignore attr-defined for compat import (#43301)`
07fdfd7ddb	2024-10-23	`Bump ''black'' to ''24.10.0'' (#43285)`
15b41b458c	2024-10-22	`feat: Add supported hooks to OpenLineage docs (#41958)`
e7d83ddcc4	2024-10-22	`nit: remove taskgroup's tooltip from OL's AirflowJobFacet (#43152)`
1f0bba2e13	2024-10-18	`feat(providers/openlineage): Use asset in common provider (#43111)`
59cf3efb8b	2024-10-15	`require 1.2.1 common.compat for openlineage provider (#43039)`
857ca4c06c	2024-10-09	`Split providers out of the main "airflow/" tree into a UV workspace project (#42505)`

1.12.2¶

Latest change: 2024-10-09

Commit	Committed	Subject
2bb8628463	2024-10-09	`Prepare docs for Oct 1st adhoc wave of providers (#42862)`
a5ffbbda17	2024-10-09	`Standard provider bash operator (#42252)`
63ff22f403	2024-10-08	`Drop python3.8 support core and providers (#42766)`
ede7cb27fd	2024-09-30	`Rename dataset related python variable names to asset (#41348)`

1.12.1¶

Latest change: 2024-09-27

Commit	Committed	Subject
bcee926d49	2024-09-27	`Prepare docs for Sep 2nd adhoc wave of providers (#42519)`
84e8cdf674	2024-09-26	`fix: OL dag start event not being emitted (#42448)`
ffff0e8b33	2024-09-23	`Fix typo in error stack trace formatting for clearer output (#42017)`

1.12.0¶

Latest change: 2024-09-21

Commit	Committed	Subject
7628d47d04	2024-09-21	`Prepare docs for Sep 1st wave of providers (#42387)`
e05c0358af	2024-09-10	`chore: bump OL provider dependencies versions (#42059)`
aa23bfdbc7	2024-09-02	`feat: notify about potential serialization failures when sending DagRun, don't serialize unnecessary params, guard listener for exceptions (#41690)`
8640f3e397	2024-09-02	`move to dag_run.logical_date from execution date in OpenLineage provider (#41889)`
00db98b5fe	2024-09-02	`fix: cast list to flattened string in openlineage InfoJsonEncodable (#41786)`
e8a5996891	2024-08-26	`Unify DAG schedule args and change default to None (#41453)`

1.11.0¶

Latest change: 2024-08-25

Commit	Committed	Subject
2f952909f5	2024-08-25	`Prepare docs for Aug 3rd wave of providers (#41714)`
4b42691021	2024-08-23	`feat: openlineage listener captures hook-level lineage (#41482)`
86e12a9a30	2024-08-21	`fix: replace dagTree with downstream_task_ids (#41587)`
75fb7acbac	2024-08-19	`Prepare docs for Aug 2nd wave of providers (#41559)`
0e7c757a1b	2024-08-15	`fix: get task dependencies without serializing task tree to string (#41494)`
fcbff15bda	2024-08-12	`Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)`
d12eb43960	2024-08-12	`feat: add debug facet to all OpenLineage events (#41217)`
ec0e9f28ea	2024-08-08	`feat: add fileloc to DAG info in AirflowRunFacet (#41311)`
c3286e8b54	2024-08-08	`feat: remove openlineage client deprecated from_environment() method (#41310)`
d9f7febe33	2024-08-08	`chore: remove openlineage deprecation warnings (#41284)`
a7353d200e	2024-08-05	`fix: return empty data instead of None when OpenLineage on_start method is missing (#41268)`

1.10.0¶

Latest change: 2024-08-03

Commit	Committed	Subject
d23881c648	2024-08-03	`Prepare docs for Aug 1st wave of providers (#41230)`
4d27069d9c	2024-08-02	`Adjust default extractor's on_failure detection for airflow 2.10 fix (#41094)`
4ab0183cfa	2024-07-28	`Add AirflowRun on COMPLETE/FAIL events (#40996)`
277e746fa4	2024-07-26	`openlineage: update docs on openlineage methods (#41051)`
592eafa82a	2024-07-26	`openlineage, docs: add missing execution_timeout conf value (#41042)`
10f250347d	2024-07-24	`openlineage: make value of slots in attrs.define consistent across all OL usages (#40992)`
eca05550d3	2024-07-24	`Set 'slots' to True for facets used in DagRun (#40972)`
579a8b87fc	2024-07-23	`openlineage: extend custom_run_facets to also be executed on complete and fail (#40953)`
0206a4cbcf	2024-07-23	`openlineage: migrate OpenLineage provider to V2 facets. (#39530)`
9ec9eb79a0	2024-07-23	`openlineage: Add AirflowRunFacet for dag runEvents (#40854)`
8a912f9fa0	2024-07-23	`[AIP-62] Translate AIP-60 URI to OpenLineage (#40173)`
e30f8102b2	2024-07-22	`Ability to add custom facet in OpenLineage events (#38982)`
12e17d1726	2024-07-22	`openlineage: replace dt.now with airflow.utils.timezone.utcnow (#40887)`
6366204458	2024-07-17	`openlineage: add method to common.compat to not force hooks to try/except every 2.10 hook lineage call (#40812)`
985ccbc4c8	2024-07-16	`openlineage: use airflow provided getters from conf (#40790)`
37a576130b	2024-07-16	`openlineage: add config to include 'full' task info based on conf setting (#40589)`
2f0bba012e	2024-07-15	`Add TaskInstance log_url to OpenLineage facet (#40797)`
ea1812112d	2024-07-12	`openlineage: fix / add some task attributes in AirflowRunFacet (#40725)`
165b91014f	2024-07-10	`openlineage: add deferrable information to task info in airflow run facet (#40682)`
14b075b4c9	2024-07-10	`openlineage: remove deprecated parentRun facet key (#40681)`

1.9.1¶

Latest change: 2024-07-09

Commit	Committed	Subject
09a7bd1d58	2024-07-09	`Prepare docs 1st wave July 2024 (#40644)`
1dc582dba3	2024-07-05	`fix openlineage parsing dag tree with MappedOperator (#40621)`

1.9.0¶

Latest change: 2024-07-02

Commit	Committed	Subject
0fae73dc45	2024-07-02	`Update docs for RC2 openlineage provider (#40551)`
57fb776408	2024-06-28	`fix: scheduler crashing with OL provider on airflow standalone (#40459)`
322130ed82	2024-06-27	`nit: fix logging level (#40461)`
a62bd83188	2024-06-27	`Enable enforcing pydocstyle rule D213 in ruff. (#40448)`
d1a8d3f916	2024-06-24	`fix: provide stack trace under proper key in OL facet (#40372)`
a1e6e598ed	2024-06-24	`Revert "fix: scheduler crashing with OL provider on airflow standalone (#40353)" (#40402)`
5df3c4efab	2024-06-23	`Add few removed Task properties in AirflowRunFacet (#40371)`
6e5ae26382	2024-06-22	`Prepare docs 2nd wave June 2024 (#40273)`
fbcee8d01b	2024-06-21	`fix: scheduler crashing with OL provider on airflow standalone (#40353)`
4fbdd07c13	2024-06-20	`fix: sqa deprecations for airflow providers (#39293)`
1a8d12ffe5	2024-06-14	`openlineage: execute extraction and message sending in separate process (#40078)`
6f4098487d	2024-06-14	`openlineage, redshift: do not call DB for schemas below Airflow 2.10 (#40197)`
fa65a20d4a	2024-06-13	`local task job: add timeout, to not kill on_task_instance_success listener prematurely (#39890)`
f0b51cdacc	2024-06-12	`openlineage: add some debug logging around sql parser call sites (#40200)`
1a613030e6	2024-06-06	`Add task SLA and queued datetime information to AirflowRunFacet (#40091)`
c202c07f67	2024-06-05	`Introduce AirflowJobFacet and AirflowStateRunFacet (#39520)`
aba8def5f3	2024-05-29	`Add error stacktrace to OpenLineage task event (#39813)`
22305477bb	2024-05-28	`Use UUIDv7 for OpenLineage runIds (#39889)`

1.8.0¶

Latest change: 2024-05-26

Commit	Committed	Subject
34500f3a2f	2024-05-26	`Prepare docs 3rd wave May 2024 (#39738)`
9532cc7a6c	2024-05-21	`fix: Prevent error when extractor can't be imported (#39736)`
b7671ef5ab	2024-05-21	`Re-configure ORM in spawned OpenLineage process in scheduler. (#39735)`
a81504e316	2024-05-21	`chore: Update conf retrieval docstring and adjust pool_size (#39721)`
4ee46b984d	2024-05-20	`Remove 'openlineage.common' dependencies in Google and Snowflake providers. (#39614)`
0b698a852b	2024-05-17	`Add missing 'dag_state_change_process_pool_size' in 'provider.yaml'. (#39674)`
4de79a0f6b	2024-05-17	`feat: Add custom provider runtime checks (#39609)`
81a82d8481	2024-05-15	`Run unit tests for Providers with airflow installed as package. (#39513)`
d529ec8d45	2024-05-15	`Use 'ProcessPoolExecutor' over 'ThreadPoolExecutor'. (#39235)`
8bc6c32366	2024-05-13	`chore: Add more OpenLineage logs to facilitate debugging (#39136)`
2b1a2f8d56	2024-05-11	`Reapply templates for all providers (#39554)`
2c05187b07	2024-05-10	`Faster 'airflow_version' imports (#39552)`
3938f71dfa	2024-05-08	`Scheduler to handle incrementing of try_number (#39336)`
73918925ed	2024-05-08	`Simplify 'airflow_version' imports (#39497)`
02ce7f1f58	2024-05-08	`openlineage: notify that logged exception was caught (#39493)`

1.7.1¶

Latest change: 2024-05-01

Commit	Committed	Subject
fe4605a10e	2024-05-01	`Prepare docs 1st wave May 2024 (#39328)`
ecc5af70eb	2024-04-22	`openlineage, snowflake: do not run external queries for Snowflake (#39113)`
09e938a2a7	2024-04-16	`Fix OpenLineage provide plugin examples (#39029)`

1.7.0¶

Latest change: 2024-04-10

Commit	Committed	Subject
5fa80b6aea	2024-04-10	`Prepare docs 1st wave (RC1) April 2024 (#38863)`
093ab7e755	2024-04-08	`Add lineage_job_namespace and lineage_job_name OpenLineage macros (#38829)`
7d9737154c	2024-04-07	`fix: try002 for provider openlineage (#38806)`
ecd69955f9	2024-04-04	`fix: Remove redundant operator information from facets (#38264)`
a2f5307fd0	2024-03-29	`Brings back mypy-checks (#38597)`
714a933479	2024-03-26	`openlineage: add 'opt-in' option (#37725)`
9c4e333f5b	2024-03-25	`fix: disabled_for_operators now stops whole event emission (#38033)`
87faf3144f	2024-03-22	`docs(openlineage): fix quotation around openlineage transport value (#38378)`
c6bc052980	2024-03-19	`Add default for 'task' on TaskInstance / fix attrs on TaskInstancePydantic (#37854)`
0a74928894	2024-03-18	`Bump ruff to 0.3.3 (#38240)`
c0b849ad2b	2024-03-11	`Avoid use of 'assert' outside of the tests (#37718)`
ea5238a81b	2024-03-08	`fix: Add fallbacks when retrieving Airflow configuration to avoid errors being raised (#37994)`
3f52790d42	2024-03-06	`Resolve G004: Logging statement uses f-string (#37873)`
2852976ea6	2024-03-05	`fix: Fix parent id macro and remove unused utils (#37877)`

1.6.0¶

Latest change: 2024-03-04

Commit	Committed	Subject
83316b8158	2024-03-04	`Prepare docs 1st wave (RC1) March 2024 (#37876)`
9848954e78	2024-03-01	`feat: Add OpenLineage metrics for event_size and extraction time (#37797)`
e358bb2253	2024-02-29	`tests: Add OpenLineage test cases for File to Dataset conversion (#37791)`
4938ac04b6	2024-02-29	`feat: Add OpenLineage support for File and User Airflow's lineage entities (#37744)`
fb6511212e	2024-02-28	`docs: Update whole OpenLineage Provider docs. (#37620)`
5289140a03	2024-02-22	`chore: Update comments and logging in OpenLineage ExtractorManager (#37622)`
1851a71278	2024-02-19	`[OpenLineage] Add support for JobTypeJobFacet properties. (#37255)`
5a0be392e6	2024-02-16	`Add comment about versions updated by release manager (#37488)`
61f0adf0c7	2024-02-14	`fix: Check if operator is disabled in DefaultExtractor.extract_on_complete (#37392)`
326f9789cd	2024-02-10	`Follow D401 style in openlineage, slack, and tableau providers (#37312)`
674ea5dd9e	2024-01-30	`fix static checks for openlineage provider (#37092)`
c81be6f78c	2024-01-30	`Fix macros jinja template example (#36222)`

1.5.0¶

Latest change: 2024-01-26

Commit	Committed	Subject
cead3da4a6	2024-01-26	`Add docs for RC2 wave of providers for 2nd round of Jan 2024 (#37019)`
2b4da0101f	2024-01-22	`Prepare docs 2nd wave of Providers January 2024 (#36945)`
95a83102e8	2024-01-09	`feat: Add dag_id when generating OpenLineage run_id for task instance. (#36659)`

1.4.0¶

Latest change: 2024-01-07

Commit	Committed	Subject
19ebcac239	2024-01-07	`Prepare docs 1st wave of Providers January 2024 (#36640)`
44b97e1687	2024-01-04	`Add OpenLineage support for Redshift SQL. (#35794)`
f0e3e612ba	2024-01-03	`feat: Add openlineage docs ext with list of supported operators and hooks. (#36311)`
6937ae7647	2023-12-30	`Speed up autocompletion of Breeze by simplifying provider state (#36499)`

1.3.1¶

Latest change: 2023-12-23

Commit	Committed	Subject
b15d5578da	2023-12-23	`Re-apply updated version numbers to 2nd wave of providers in December (#36380)`
f5883d6e7b	2023-12-23	`Prepare 2nd wave of providers in December (#36373)`
fcb527ae94	2023-12-22	`Fix typo. (#36362)`

1.3.0¶

Latest change: 2023-12-08

Commit	Committed	Subject
999b70178a	2023-12-08	`Prepare docs 1st wave of Providers December 2023 (#36112)`
fba682b1a5	2023-12-06	`Remove ClassVar annotations. (#36084)`
acd95a5ef1	2023-12-05	`feat: Add parent_run_id for COMPLETE and FAIL events (#36067)`
35a1b7a63a	2023-12-04	`fix: Repair run_id for OpenLineage FAIL events (#36051)`
0b23d5601c	2023-11-24	`Prepare docs 2nd wave of Providers November 2023 (#35836)`
ecbf02386a	2023-11-21	`Add basic metrics to stats collector. (#35368)`
99534e47f3	2023-11-19	`Use reproducible builds for provider packages (#35693)`
99df205f42	2023-11-16	`Fix and reapply templates for provider documentation (#35686)`

1.2.1¶

Latest change: 2023-11-08

Commit	Committed	Subject
1b059c57d6	2023-11-08	`Prepare docs 1st wave of Providers November 2023 (#35537)`
59b32dc0a0	2023-11-06	`Fix bad regexp in mypy-providers specification in pre-commits (#35465)`
706878ec35	2023-11-04	`Remove empty lines in generated changelog (#35436)`
052e26ad47	2023-11-04	`Change security.rst to use includes in providers (#35435)`
6858ea46eb	2023-11-04	`Make schema filter uppercase in 'create_filter_clauses' (#35428)`
63cc915cd3	2023-10-31	`Switch from Black to Ruff formatter (#35287)`

1.2.0¶

Latest change: 2023-10-28

Commit	Committed	Subject
d1c58d86de	2023-10-28	`Prepare docs 3rd wave of Providers October 2023 - FIX (#35233)`
3592ff4046	2023-10-28	`Prepare docs 3rd wave of Providers October 2023 (#35187)`
0940d09859	2023-10-25	`Send column lineage from SQL operators. (#34843)`
dd7ba3cae1	2023-10-19	`Pre-upgrade 'ruff==0.0.292' changes in providers (#35053)`

1.1.1¶

Latest change: 2023-10-13

Commit	Committed	Subject
e9987d5059	2023-10-13	`Prepare docs 1st wave of Providers in October 2023 (#34916)`
73dd877961	2023-10-11	`Adjust log levels in OpenLineage provider (#34801)`

1.1.0¶

Latest change: 2023-09-08

Commit	Committed	Subject
21990ed894	2023-09-08	`Prepare docs for 09 2023 - 1st wave of Providers (#34201)`
5eea4e632c	2023-09-07	`Add OpenLineage support for DBT Cloud. (#33959)`
e403c74524	2023-09-06	`Fix import in 'get_custom_facets'. (#34122)`
875387afa5	2023-09-01	`Refactor unneeded jumps in providers (#33833)`
b4d4f55b47	2023-08-31	`Refactor: Replace lambdas with comprehensions in providers (#33771)`
0d49d1fed9	2023-08-29	`Allow to disable openlineage at operator level (#33685)`
9d8c77e447	2023-08-27	`Improve modules import in Airflow providers by some of them into a type-checking block (#33754)`

1.0.2¶

Latest change: 2023-08-26

Commit	Committed	Subject
c077d19060	2023-08-26	`Prepare docs for Aug 2023 3rd wave of Providers (#33730)`
626d3daa9b	2023-08-24	`Add OpenLineage support for Trino. (#32910)`
ed5e70c6d6	2023-08-22	`openlineage: finish user guide (#33610)`
1cdd82391e	2023-08-21	`Simplify conditions on len() in other providers (#33569)`
abef61ff3d	2023-08-20	`Replace repr() with proper formatting (#33520)`
6d3b71c333	2023-08-19	`openlineage: don't run task instance listener in executor (#33366)`
8e738cd0ad	2023-08-15	`openlineage: do not try to redact Proxy objects from deprecated config (#33393)`
23d5076635	2023-08-13	`openlineage: defensively check for provided datetimes in listener (#33343)`

1.0.1¶

Latest change: 2023-08-05

Commit	Committed	Subject
60677b0ba3	2023-08-05	`Prepare docs for Aug 2023 1st wave of Providers (#33128)`
bdc10a5ff6	2023-08-04	`Move openlineage configuration to provider (#33124)`
11ff650e1b	2023-08-04	`openlineage: disable running listener if not configured (#33120)`
e10aa6ae6a	2023-08-04	`openlineage, bigquery: add openlineage method support for BigQueryExecuteQueryOperator (#31293)`
2a39914cbd	2023-08-01	`Don't use database as fallback when no schema parsed. (#32959)`
f9cddf332d	2023-07-31	`openlineage,gcs: use proper name for openlineage methods (#32956)`

1.0.0¶

Latest change: 2023-07-29

Commit	Committed	Subject
d06b7af69a	2023-07-29	`Prepare docs for July 2023 3rd wave of Providers (#32875)`
0924389a87	2023-07-28	`Fix MIN_AIRFLOW_VERSION_EXCEPTIONS for openlineage (#32909)`
5c8223c335	2023-07-28	`Bump common-sql version in provider (#32907)`
8a0f410010	2023-07-28	`Update openlineage provider to min version of airflow 2.7.0 (#32882)`
b73366799d	2023-07-27	`openlineage, gcs: add openlineage methods for GcsToGcsOperator (#31350)`
9194144dab	2023-07-27	`Replace Ruff setting known-third-party with namespace-packages (#32873)`
73b90c48b1	2023-07-21	`Allow configuration to be contributed by providers (#32604)`
3878fe6fab	2023-07-05	`Remove spurious headers for provider changelogs (#32373)`
ee4a838d49	2023-07-05	`Pass SQLAlchemy engine to construct information schema query. (#32371)`
1240dcc167	2023-07-05	`D205 Support - Providers: GRPC to Oracle (inclusive) (#32357)`
65fad4affc	2023-07-04	`Change default schema behaviour in SQLParser. (#32347)`
f2e2125b07	2023-06-29	`openlineage, common.sql: provide OL SQL parser as internal OpenLineage provider API (#31398)`
3d8e214e90	2023-06-28	`openlineage, docs: tips for OpenLineage method implementation (#31817)`
09d4718d3a	2023-06-27	`Improve provider documentation and README structure (#32125)`
1d564adc1c	2023-06-20	`set contributor note in CHANGELOG.rst openlienage (#32018)`
ebd7b0eb53	2023-06-13	`openlineage: fix typing errors produced by bumping version, bump minimum version to 0.28, remove outdated warnings (#31874)`
6f8cd65bde	2023-06-06	`Limit openlineage-integration-common until breaking change is fixed (#31739)`
9276310a43	2023-06-05	`Improve docstrings in providers (#31681)`
a59076eaee	2023-06-02	`Add D400 pydocstyle check - Providers (#31427)`
abea189022	2023-05-18	`Use '__version__' in providers not 'version' (#31393)`
f5aed58d9f	2023-05-18	`Fixing circular import error in providers caused by airflow version check (#31379)`
6b21e4b88c	2023-05-13	`Bring back detection of implicit single-line string concatenation (#31270)`
981afe2a4f	2023-05-12	`openlineage: add extractors for python and bash operators (#30713)`
51603efbf7	2023-05-04	`Allow configuring OpenLineage client from Airflow config. (#30735)`
eef5bc7f16	2023-05-03	`Add full automation for min Airflow version for providers (#30994)`
2f570c2bf7	2023-04-20	`Fix when OpenLineage plugins has listener disabled. (#30708)`
cbde23e6bc	2023-04-17	`Upgrade to MyPy 1.2.0 (#30687)`
6a6455ad1c	2023-04-17	`Correctly pass a type to attrs.has() (#30677)`
8d81963c01	2023-04-16	`Workaround type-incompatibility with new attrs in openlineage (#30674)`
55963de61e	2023-04-14	`First commit of OpenLineage provider. (#29940)`