Configuration Reference¶

This page contains the list of all available Airflow configurations for the apache-airflow-providers-common-io provider that can be set in the airflow.cfg file or using environment variables.

Note

The configuration embedded in provider packages started to be used as of Airflow 2.7.0. Previously the configuration was described and configured in the Airflow core package - so if you are using Airflow below 2.7.0, look at Airflow documentation for the list of available configuration options that were available in Airflow core.

Note

For more information see Setting Configuration Options.

[common.io]¶

Common IO configuration section

xcom_objectstorage_compression¶

Added in version 1.3.0.

Compression algorithm to use when storing XComs in object storage. Supported algorithms are a.o.: snappy, zip, gzip, bz2, and lzma. If not specified, no compression will be used. Note that the compression algorithm must be available in the Python installation (e.g. python-snappy for snappy). Zip, gz, bz2 are available by default.

Type:: string
Default:: ''
Environment Variable:: AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_COMPRESSION
Example:: gz

xcom_objectstorage_path¶

Added in version 1.3.0.

Path to a location on object storage where XComs can be stored in url format.

Type:: string
Default:: ''
Environment Variable:: AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_PATH
Example:: s3://conn_id@bucket/path

xcom_objectstorage_threshold¶

Added in version 1.3.0.

Threshold in bytes for storing XComs in object storage. -1 means always store in the database. 0 means always store in object storage. Any positive number means it will be stored in object storage if the size of the value is greater than the threshold.

Type:: integer
Default:: -1
Environment Variable:: AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_THRESHOLD
Example:: 1000000