• Home
• airflow.providers.snowflake
• airflow.providers.snowflake.operators
• airflow.providers.snowflake.operators.snowpark_containers

↑↓ Navigate ⏎ Select Esc Close

airflow.providers.snowflake.operators.snowpark_containers

Attributes

TERMINAL_STATUSES
NON_TERMINAL_STATUSES

Classes

SnowparkContainerJobStatus	Statuses of a Snowpark Container Services.
SnowparkContainerJobOperator	Execute a job on Snowpark Container Services.

Module Contents

class airflow.providers.snowflake.operators.snowpark_containers.SnowparkContainerJobStatus

Bases: str, enum.Enum

Statuses of a Snowpark Container Services.

PENDING = 'PENDING'

RUNNING = 'RUNNING'

CANCELLING = 'CANCELLING'

SUSPENDING = 'SUSPENDING'

DELETING = 'DELETING'

DONE = 'DONE'

FAILED = 'FAILED'

CANCELLED = 'CANCELLED'

INTERNAL_ERROR = 'INTERNAL_ERROR'

airflow.providers.snowflake.operators.snowpark_containers.TERMINAL_STATUSES: frozenset[SnowparkContainerJobStatus]

airflow.providers.snowflake.operators.snowpark_containers.NON_TERMINAL_STATUSES: frozenset[SnowparkContainerJobStatus]

class airflow.providers.snowflake.operators.snowpark_containers.SnowparkContainerJobOperator(*, compute_pool, container_name, spec=None, spec_stage=None, spec_text=None, name=None, query_warehouse=None, replicas=1, wait_for_completion=True, drop_on_completion=True, poll_interval=10, snowflake_conn_id='snowflake_default', database=None, schema=None, role=None, warehouse=None, **kwargs)

Bases: airflow.providers.common.compat.standard.operators.BaseOperator

Execute a job on Snowpark Container Services.

Submits a container job to a compute pool via EXECUTE JOB SERVICE, optionally polls for completion, retrieves container logs, and drops the job service on success.

See also

Snowpark Container Services

Parameters:

• compute_pool (str) – name of the compute pool to run the job on

• container_name (str) – container name as defined in the service specification file, used for retrieving container logs

• spec (str | None) – spec filename on the stage (e.g. 'spec.yaml'). Must be provided together with spec_stage

• spec_stage (str | None) – stage where the spec file is stored (e.g. '@my_stage'). Must be provided together with spec

• spec_text (str | None) – inline YAML spec text, as an alternative to spec/spec_stage. The text is wrapped in $$ delimiters automatically

• name (str | None) – (Optional) job service name. If not provided, Snowflake auto-generates a name

• query_warehouse (str | None) – (Optional) warehouse for SQL queries run inside the container. This is separate from the warehouse parameter used by the operator’s own SQL commands

• replicas (int) – (Optional) number of job replicas to run. (default value: 1)

• wait_for_completion (bool) – poll until the job reaches a terminal state. When disabled, the job is submitted and the operator returns immediately. (default value: True)

• drop_on_completion (bool) – drop the job service after the job finishes successfully. Failed jobs are not dropped, allowing inspection in Snowflake. (default value: True)

• poll_interval (int) – the interval in seconds to poll the query status. (default value: 10)

• snowflake_conn_id (str) – Reference to Snowflake connection id

• database (str | None) – name of database (will overwrite database defined in connection)

• schema (str | None) – name of schema (will overwrite schema defined in connection)

• role (str | None) – name of role (will overwrite any role defined in connection’s extra JSON)

• warehouse (str | None) – name of warehouse (will overwrite any warehouse defined in the connection’s extra JSON). Used for the operator’s own SQL commands, not for the container’s queries

template_fields: collections.abc.Sequence[str] = ('compute_pool', 'spec', 'spec_stage', 'container_name', 'spec_text', 'name', 'query_warehouse',...

compute_pool

container_name

spec = None

spec_stage = None

spec_text = None

name = None

query_warehouse = None

replicas = 1

wait_for_completion = True

drop_on_completion = True

poll_interval = 10

snowflake_conn_id = 'snowflake_default'

database = None

schema = None

role = None

warehouse = None

job_name: str | None = None

on_kill()

Drop the running service on task kill.

execute(context)

Submit and optionally wait for a Snowpark Container Services job.

Previous Next

Was this entry helpful?