airflow.providers.amazon.aws.hooks.athena¶
This module contains AWS Athena hook.
Attributes¶
Classes¶
Interact with Amazon Athena. |
Functions¶
|
Module Contents¶
- airflow.providers.amazon.aws.hooks.athena.MULTI_LINE_QUERY_LOG_PREFIX = Multiline-String[source]¶
Show Value
""" """
- class airflow.providers.amazon.aws.hooks.athena.AthenaHook(*args, log_query=True, **kwargs)[source]¶
Bases:
airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHookInteract with Amazon Athena.
Provide thick wrapper around
boto3.client("athena").- Parameters:
log_query (bool) – Whether to log athena query and other execution params when it’s executed. Defaults to True.
Additional arguments (such as
aws_conn_id) may be specified and are passed down to the underlying AwsBaseHook.- run_query(query, query_context, result_configuration=None, client_request_token=None, workgroup='primary')[source]¶
Run a Trino/Presto query on Athena with provided config.
- Parameters:
query (str) – Trino/Presto query to run.
query_context (dict[str, str]) – Context in which query need to be run.
result_configuration (dict[str, Any] | None) – Dict with path to store results in and config related to encryption. Leave it unset for a workgroup with managed query results, which keeps the results in Athena.
client_request_token (str | None) – Unique token created by user to avoid multiple executions of same query.
workgroup (str) – Athena workgroup name, when not specified, will be
'primary'.
- Returns:
Submitted query execution ID.
- Return type:
- get_query_info(query_execution_id, use_cache=False)[source]¶
Get information about a single execution of a query.
See also
- check_query_status(query_execution_id, use_cache=False)[source]¶
Fetch the state of a submitted query.
See also
- get_state_change_reason(query_execution_id, use_cache=False)[source]¶
Fetch the reason for a state change (e.g. error message). Returns None or reason string.
See also
- Parameters:
query_execution_id (str) – Id of submitted athena query
- get_query_results(query_execution_id, next_token_id=None, max_results=1000)[source]¶
Fetch submitted query results.
See also
- Parameters:
- Returns:
None if the query is in intermediate, failed, or cancelled state. Otherwise a dict of query outputs.
- Return type:
dict | None
- get_query_results_paginator(query_execution_id, max_items=None, page_size=None, starting_token=None)[source]¶
Fetch submitted Athena query results.
See also
- Parameters:
- Returns:
None if the query is in intermediate, failed, or cancelled state. Otherwise a paginator to iterate through pages of results.
- Return type:
botocore.paginate.PageIterator | None
Call :meth`.build_full_result()` on the returned paginator to get all results at once.
- poll_query_status(query_execution_id, max_polling_attempts=None, sleep_time=None)[source]¶
Poll the state of a submitted query until it reaches final state.
- Parameters:
- Returns:
One of the final states
- Return type:
str | None
- get_output_location(query_execution_id)[source]¶
Get the output location of the query results in S3 URI format.
Returns None when the workgroup has managed query results enabled, since Athena stores those results itself instead of in S3.
See also
- Parameters:
query_execution_id (str) – Id of submitted athena query
- stop_query(query_execution_id)[source]¶
Cancel the submitted query.
See also
- Parameters:
query_execution_id (str) – Id of submitted athena query
- start_spark_calculation(*, session_id, code_block, description=None, client_request_token=None)[source]¶
Start an Athena Spark calculation execution.
- Parameters:
- Returns:
CalculationExecutionId
- Return type:
- get_spark_calculation_info(calculation_execution_id, use_cache=False)[source]¶
Get information about a single Athena Spark calculation execution.
- check_spark_calculation_status(calculation_execution_id, use_cache=False)[source]¶
Fetch the state of a submitted Athena Spark calculation execution.
- get_spark_calculation_state_change_reason(calculation_execution_id, use_cache=False)[source]¶
Fetch the reason for an Athena Spark calculation state change, such as an error message.