airflow.providers.apache.arrow.hooks.adbc¶

Classes¶

AdbcHook

General-purpose Airflow hook for interacting with databases via the Arrow Database Connectivity (ADBC) standard.

Functions¶

`fetch_all_handler`(cursor)	Return results for DbApiHook.run().
`replace_placeholders`(sql, placeholder)

Module Contents¶

airflow.providers.apache.arrow.hooks.adbc.fetch_all_handler(cursor)[source]¶

Return results for DbApiHook.run().

airflow.providers.apache.arrow.hooks.adbc.replace_placeholders(sql, placeholder)[source]¶

class airflow.providers.apache.arrow.hooks.adbc.AdbcHook(*args, schema=None, log_sql=True, **kwargs)[source]¶

Bases: airflow.providers.common.sql.hooks.sql.DbApiHook

General-purpose Airflow hook for interacting with databases via the Arrow Database Connectivity (ADBC) standard.

This hook enables connections to any database supported by an ADBC driver, using the Python ADBC driver manager. It provides methods for executing SQL queries, inserting rows in bulk, and handling Arrow-native data transfers.

Key Features:

Supports chunked and batched inserts using Apache Arrow RecordBatches for efficient data transfer.
Discovers and loads ADBC drivers dynamically based on connection extras or naming conventions.
Handles dialect-specific connection URIs and driver entrypoints.
Integrates with Airflow’s connection system (conn_id, extras, etc.).
Provides custom placeholder replacement for parameterized SQL queries.
Supports both native Arrow binding and DBAPI executemany for inserts.
Exposes configuration via connection extras: driver, entrypoint, db_kwargs, conn_kwargs, dialect.

Connection Extras:

driver: Name of the ADBC driver to use (e.g., “adbc_driver_postgresql”).
entrypoint: Optional Python entrypoint for the driver.
db_kwargs: Driver-specific database initialization options passed to AdbcDatabase. Keys and value types are defined by the driver (e.g., "username", "password" for the PostgreSQL driver). Do not put ADBC connection options here — they belong in conn_kwargs.
conn_kwargs: ADBC connection options as string key-value pairs. Keys must use the canonical dotted ADBC option names, for example: "adbc.connection.autocommit": "true", "adbc.connection.read_only": "true", "adbc.connection.current_catalog": "my_catalog", "adbc.connection.current_db_schema": "my_schema". Short names such as autocommit or read_only are not recognized by ADBC drivers and will raise NotSupportedError.
dialect: SQL dialect name (default: “default”).

Example usage:

hook = AdbcHook(adbc_conn_id=”my_adbc_conn”) records = hook.get_records(“SELECT * FROM my_table”)

For more details, see:

Apache Arrow ADBC Python API: https://arrow.apache.org/adbc/current/python/api/adbc_driver_manager.html
Airflow SQL hooks: https://airflow.apache.org/docs/apache-airflow/stable/howto/custom-operator.html#hooks

conn_name_attr = 'adbc_conn_id'[source]¶

default_conn_name = 'adbc_default'[source]¶

conn_type = 'adbc'[source]¶

hook_name = 'ADBC Connection'[source]¶

supports_autocommit = True[source]¶

classmethod get_ui_field_behaviour()[source]¶

Get custom field behaviour.

property uri: str[source]¶

property driver: str[source]¶

property entrypoint: str | None[source]¶

property db_kwargs: dict[source]¶

property conn_kwargs: dict[source]¶

property dialect_name: str[source]¶

get_conn()[source]¶

Return a connection object.

set_autocommit(conn, autocommit)[source]¶

Set autocommit on the ADBC connection.

The DBAPI attribute conn.autocommit has no effect on the underlying ADBC driver; the real lever is the adbc.connection.autocommit option. This override applies the option at the driver level, then calls super() to keep the Python-level bookkeeping attribute in sync so that the base class get_autocommit() and run() commit-guard behave correctly.

get_records(sql, parameters=None)[source]¶

Execute the sql and return a set of records.

Parameters:

sql (str | list[str]) – the sql statement to be executed (str) or a list of sql statements to execute
parameters (collections.abc.Iterable | collections.abc.Mapping[str, Any] | None) – The parameters to render the SQL query with.

insert_rows(table, rows, target_fields=None, commit_every=1000, replace=False, *, executemany=False, fast_executemany=False, autocommit=False, **kwargs)[source]¶

Insert a collection of tuples into a table.

Rows are inserted in chunks, each chunk (of size commit_every) is done in a new transaction.

Parameters:

table – Name of the target table
rows – The rows to insert into the table
target_fields – The names of the columns to fill in the table
commit_every – The maximum number of rows to insert in one transaction. Set to 0 to insert all rows in one transaction.
replace – Whether to replace instead of insert
executemany – If True, all rows are inserted at once in chunks defined by the commit_every parameter. This only works if all rows have same number of column names, but leads to better performance.
fast_executemany – If True, the fast_executemany parameter will be set on the cursor used by executemany which leads to better performance, if supported by driver.
autocommit – What to set the connection’s autocommit setting to before executing the query.