Configuration Reference¶

This page contains the list of all available Airflow configurations for the apache-airflow-providers-opensearch provider that can be set in the airflow.cfg file or using environment variables.

Note

For more information see Setting Configuration Options.

[opensearch]¶

end_of_log_mark¶

Used to mark the end of a log stream for a task

Type:: string
Default:: end_of_log
Environment Variable:: AIRFLOW__OPENSEARCH__END_OF_LOG_MARK

host¶

Opensearch host

Type:: string
Default:: ''
Environment Variable:: AIRFLOW__OPENSEARCH__HOST

host_field¶

The field where host name is stored (normally either host or host.name)

Type:: string
Default:: host
Environment Variable:: AIRFLOW__OPENSEARCH__HOST_FIELD

index_patterns¶

Comma separated list of index patterns to use when searching for logs (default: _all). The index_patterns_callable takes precedence over this.

Type:: string
Default:: _all
Environment Variable:: AIRFLOW__OPENSEARCH__INDEX_PATTERNS
Example:: something-*

index_patterns_callable¶

A string representing the full path to the Python callable path which accept TI object and return comma separated list of index patterns. This will takes precedence over index_patterns.

Type:: string
Default:: ''
Environment Variable:: AIRFLOW__OPENSEARCH__INDEX_PATTERNS_CALLABLE
Example:: module.callable

json_fields¶

Log fields to also attach to the json output, if enabled

Type:: string
Default:: asctime, filename, lineno, levelname, message
Environment Variable:: AIRFLOW__OPENSEARCH__JSON_FIELDS

json_format¶

Instead of the default log formatter, write the log lines as JSON

Type:: string
Default:: False
Environment Variable:: AIRFLOW__OPENSEARCH__JSON_FORMAT

log_id_template¶

Format of the log_id, which is used to query for a given tasks logs

Type:: string
Default:: {dag_id}-{task_id}-{run_id}-{map_index}-{try_number}
Environment Variable:: AIRFLOW__OPENSEARCH__LOG_ID_TEMPLATE

offset_field¶

The field where offset is stored (normally either offset or log.offset)

Type:: string
Default:: offset
Environment Variable:: AIRFLOW__OPENSEARCH__OFFSET_FIELD

password¶

The password for connecting to Opensearch

Type:

string

Default:

''

Environment Variables:

AIRFLOW__OPENSEARCH__PASSWORD

AIRFLOW__OPENSEARCH__PASSWORD_CMD

AIRFLOW__OPENSEARCH__PASSWORD_SECRET

port¶

The port number of Opensearch host

Type:: integer
Default:: ''
Environment Variable:: AIRFLOW__OPENSEARCH__PORT

username¶

The username for connecting to Opensearch

Type:

string

Default:

''

Environment Variables:

AIRFLOW__OPENSEARCH__USERNAME

AIRFLOW__OPENSEARCH__USERNAME_CMD

AIRFLOW__OPENSEARCH__USERNAME_SECRET

write_stdout¶

Write the task logs to the stdout of the worker, rather than the default files

Type:: string
Default:: False
Environment Variable:: AIRFLOW__OPENSEARCH__WRITE_STDOUT

[opensearch_configs]¶

ca_certs¶

Type:: string
Default:: ''
Environment Variable:: AIRFLOW__OPENSEARCH_CONFIGS__CA_CERTS

http_compress¶

Type:: string
Default:: False
Environment Variable:: AIRFLOW__OPENSEARCH_CONFIGS__HTTP_COMPRESS

ssl_assert_hostname¶

Type:: string
Default:: False
Environment Variable:: AIRFLOW__OPENSEARCH_CONFIGS__SSL_ASSERT_HOSTNAME

ssl_show_warn¶

Type:: string
Default:: False
Environment Variable:: AIRFLOW__OPENSEARCH_CONFIGS__SSL_SHOW_WARN

use_ssl¶

Type:: string
Default:: False
Environment Variable:: AIRFLOW__OPENSEARCH_CONFIGS__USE_SSL

verify_certs¶

Type:: string
Default:: False
Environment Variable:: AIRFLOW__OPENSEARCH_CONFIGS__VERIFY_CERTS