airflow.providers.edge3.cli.worker¶

Attributes¶

`logger`
`base_log_folder`
`push_logs`
`push_log_chunk_size`
`setproctitle`

Classes¶

EdgeWorker

Runner instance which executes the Edge Worker.

Module Contents¶

airflow.providers.edge3.cli.worker.logger[source]¶

airflow.providers.edge3.cli.worker.base_log_folder[source]¶

airflow.providers.edge3.cli.worker.push_logs = True[source]¶

airflow.providers.edge3.cli.worker.push_log_chunk_size[source]¶

airflow.providers.edge3.cli.worker.setproctitle[source]¶

class airflow.providers.edge3.cli.worker.EdgeWorker(pid_file_path, hostname, queues, concurrency, job_poll_interval, heartbeat_interval, daemon=False)[source]¶

Runner instance which executes the Edge Worker.

jobs: list[airflow.providers.edge3.cli.dataclasses.Job] = [][source]¶: List of jobs that the worker is running currently.

drain: bool = False[source]¶: Flag if job processing should be completed and no new jobs fetched for a graceful stop/shutdown.

maintenance_mode: bool = False[source]¶: Flag if job processing should be completed and no new jobs fetched for maintenance mode.

maintenance_comments: str | None = None[source]¶: Comments for maintenance mode.

background_tasks: set[asyncio.Task][source]¶

pid_file_path[source]¶

job_poll_interval[source]¶

hb_interval[source]¶

hostname[source]¶

queues[source]¶

concurrency[source]¶

daemon = False[source]¶

property free_concurrency: int[source]¶

Calculate the free concurrency of the worker.

signal_status()[source]¶

signal_drain()[source]¶

shutdown_handler()[source]¶

async start()[source]¶: Start the execution in a loop until terminated.

async loop()[source]¶: Run a loop of scheduling and monitoring tasks.

async fetch_and_run_job()[source]¶

Fetch, start and monitor a new job.

async heartbeat(new_maintenance_comments=None)[source]¶

Report liveness state of worker to central site with stats.

async interruptible_sleep()[source]¶: Sleeps but stops sleeping if drain is made or some job completed.