airflow.providers.apache.kafka.hooks.consume

Module Contents

Classes

KafkaConsumerHook

A hook for creating a Kafka Consumer.

Functions

error_callback(err)

Handle kafka errors.
exception airflow.providers.apache.kafka.hooks.consume.KafkaAuthenticationError[source]

Bases: Exception

Custom exception for Kafka authentication failures.

airflow.providers.apache.kafka.hooks.consume.error_callback(err)[source]

Handle kafka errors.

class airflow.providers.apache.kafka.hooks.consume.KafkaConsumerHook(topics, kafka_config_id=KafkaBaseHook.default_conn_name)[source]

Bases: airflow.providers.apache.kafka.hooks.base.KafkaBaseHook

A hook for creating a Kafka Consumer.

Parameters

  • kafka_config_id – The connection object to use, defaults to “kafka_default”

  • topics (collections.abc.Sequence[str]) – A list of topics to subscribe to.

get_consumer()[source]

Return a Consumer that has been subscribed to topics.

Was this entry helpful?