Configuration Reference¶

This page contains the list of all available Airflow configurations for the apache-airflow-providers-common-io provider that can be set in the airflow.cfg file or using environment variables.

Note

For more information see Setting Configuration Options.

[common.io]¶

Common IO configuration section

xcom_objectstorage_compression¶

Added in version 1.3.0.

Compression algorithm to use when storing XComs in object storage. Supported algorithms are a.o.: snappy, zip, gzip, bz2, and lzma. If not specified, no compression will be used. Note that the compression algorithm must be available in the Python installation (e.g. python-snappy for snappy). Zip, gz, bz2 are available by default.

Type:: string
Default:: ''
Environment Variable:: AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_COMPRESSION
Example:: gz

xcom_objectstorage_path¶

Added in version 1.3.0.

Path to a location on object storage where XComs can be stored in url format.

Type:: string
Default:: ''
Environment Variable:: AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_PATH
Example:: s3://conn_id@bucket/path

xcom_objectstorage_threshold¶

Added in version 1.3.0.

Threshold in bytes for storing XComs in object storage. -1 means always store in the database. 0 means always store in object storage. Any positive number means it will be stored in object storage if the size of the value is greater than the threshold.

Type:: integer
Default:: -1
Environment Variable:: AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_THRESHOLD
Example:: 1000000