airflow.providers.edge3.cli.worker¶

Attributes¶

logger

Classes¶

EdgeWorker

Runner instance which executes the Edge Worker.

Module Contents¶

airflow.providers.edge3.cli.worker.logger[source]¶

class airflow.providers.edge3.cli.worker.EdgeWorker(pid_file_path, hostname, queues, concurrency, job_poll_interval, heartbeat_interval, daemon=False)[source]¶

Runner instance which executes the Edge Worker.

jobs: list[airflow.providers.edge3.cli.dataclasses.Job] = [][source]¶: List of jobs that the worker is running currently.

last_hb: datetime.datetime | None = None[source]¶: Timestamp of last heart beat sent to server.

drain: bool = False[source]¶: Flag if job processing should be completed and no new jobs fetched for a graceful stop/shutdown.

maintenance_mode: bool = False[source]¶: Flag if job processing should be completed and no new jobs fetched for maintenance mode.

maintenance_comments: str | None = None[source]¶: Comments for maintenance mode.

edge_instance: EdgeWorker | None = None[source]¶: Singleton instance of the worker.

pid_file_path[source]¶

job_poll_interval[source]¶

hb_interval[source]¶

hostname[source]¶

queues[source]¶

concurrency[source]¶

free_concurrency[source]¶

daemon = False[source]¶

static signal_handler(sig, frame)[source]¶

shutdown_handler(sig, frame)[source]¶

start()[source]¶: Start the execution in a loop until terminated.

loop()[source]¶: Run a loop of scheduling and monitoring tasks.

fetch_job()[source]¶

Fetch and start a new job from central site.

check_running_jobs()[source]¶

Check which of the running tasks/jobs are completed and report back.

heartbeat(new_maintenance_comments=None)[source]¶

Report liveness state of worker to central site with stats.

interruptible_sleep()[source]¶: Sleeps but stops sleeping if drain is made.