HTTP Connection¶

The HTTP connection enables connections to HTTP services.

Authenticating with HTTP¶

Login and Password authentication can be used along with any authentication method using headers. Headers can be given in json format in the Extras field.

Default Connection IDs¶

The HTTP operators and hooks use http_default by default.

Configuring the Connection¶

Login (optional)

Specify the login for the http service you would like to connect too.

Password (optional)

Specify the password for the http service you would like to connect too.

Host (optional)

Specify the entire url or the base of the url for the service.

If “Use DNS SRV Lookup” is enabled, specify the DNS SRV record name instead (e.g. _http._tcp.example.com) - Note the actual host and port are resolved from DNS at request time and any value set in the Port field is ignored.

Port (optional)

Specify a port number if applicable. Ignored when SRV lookup is enabled.

Schema (optional)

Specify the service type etc: http/https.

Use DNS SRV Lookup (optional)

Treat the Host field as a DNS SRV record name and resolve the target host/port at request time.

Targets are tried in RFC 2782 order: lowest priority first, weighted random within the same priority. If a connection to a target cannot be established, the request fails over to the next target.

Resolving SRV records requires the srv extra: pip install apache-airflow-providers-http[srv].

Extra (optional)

Specify headers and default requests parameters in json format. Following default requests parameters are taken into account:

stream
cert
proxies or proxy
verify or verify_ssl
allow_redirects
max_redirects

“Use DNS SRV Lookup” above is stored as the srv_lookup key in this same Extra field, so it can also be set directly in json here, e.g. when configuring the connection via an environment variable.

When specifying the connection in environment variable you should specify it using URI syntax.

Note that all components of the URI should be URL-encoded.

For example:

export AIRFLOW_CONN_HTTP_DEFAULT='http://username:password@service.com:80/https?headers=header'

To enable SRV lookup via an environment variable, set srv_lookup in the Extra query parameter:

export AIRFLOW_CONN_HTTP_DEFAULT='http://_http._tcp.example.com/https?srv_lookup=true'