airflow.providers.edge3.models.edge_worker¶

Attributes¶

logger

Exceptions¶

EdgeWorkerVersionException

Signal a version mismatch between core and Edge Site.

Classes¶

`EdgeWorkerState`	Status of a Edge Worker instance.
`EdgeWorkerModel`	A Edge Worker instance which reports the state and health.

Functions¶

`set_metrics`(worker_name, state, jobs_active, ...)	Set metric of edge worker.
`reset_metrics`(worker_name)	Reset metrics of worker.
`get_registered_edge_hosts`([states, session])
`request_maintenance`(worker_name, maintenance_comment)	Write maintenance request to the db.
`exit_maintenance`(worker_name[, session])	Write maintenance exit to the db.
`remove_worker`(worker_name[, session])	Remove a worker that is offline or just gone from DB.
`change_maintenance_comment`(worker_name, ...[, session])	Write maintenance comment in the db.
`request_shutdown`(worker_name[, session])	Request to shutdown the edge worker.
`add_worker_queues`(worker_name, queues[, session])	Add queues to an edge worker.
`remove_worker_queues`(worker_name, queues[, session])	Remove queues from an edge worker.

Module Contents¶

airflow.providers.edge3.models.edge_worker.logger[source]¶

exception airflow.providers.edge3.models.edge_worker.EdgeWorkerVersionException[source]¶

Bases: airflow.exceptions.AirflowException

Signal a version mismatch between core and Edge Site.

class airflow.providers.edge3.models.edge_worker.EdgeWorkerState[source]¶

Bases: str, enum.Enum

Status of a Edge Worker instance.

STARTING = 'starting'[source]¶: Edge Worker is in initialization.

RUNNING = 'running'[source]¶: Edge Worker is actively running a task.

IDLE = 'idle'[source]¶: Edge Worker is active and waiting for a task.

SHUTDOWN_REQUEST = 'shutdown request'[source]¶: Request to shutdown Edge Worker is issued. It will be picked-up on the next heartbeat, tasks will drain and then worker will terminate.

TERMINATING = 'terminating'[source]¶: Edge Worker is completing work (draining running tasks) and stopping.

OFFLINE = 'offline'[source]¶: Edge Worker was shut down.

UNKNOWN = 'unknown'[source]¶: No heartbeat signal from worker for some time, Edge Worker probably down or got disconnected.

MAINTENANCE_REQUEST = 'maintenance request'[source]¶: Worker was requested to enter maintenance mode. Once worker receives this message it will pause fetching tasks and drain tasks.

MAINTENANCE_PENDING = 'maintenance pending'[source]¶: Edge Worker received the request for maintenance, waiting for tasks to finish. Once tasks are finished will move to ‘maintenance mode’.

MAINTENANCE_MODE = 'maintenance mode'[source]¶: Edge Worker is in maintenance mode. It is online but pauses fetching tasks.

MAINTENANCE_EXIT = 'maintenance exit'[source]¶: Request Worker is requested to exit maintenance mode. Once the worker receives this state it will un-pause and fetch new tasks.

OFFLINE_MAINTENANCE = 'offline maintenance'[source]¶: Worker was shut down in maintenance mode. It will be in maintenance mode when restarted.

class airflow.providers.edge3.models.edge_worker.EdgeWorkerModel(worker_name, state, queues, first_online=None, last_update=None, maintenance_comment=None)[source]¶

Bases: airflow.models.base.Base, airflow.utils.log.logging_mixin.LoggingMixin

A Edge Worker instance which reports the state and health.

__tablename__ = 'edge_worker'[source]¶

worker_name[source]¶

state[source]¶

maintenance_comment[source]¶

first_online[source]¶

last_update[source]¶

jobs_active[source]¶

jobs_taken[source]¶

jobs_success[source]¶

jobs_failed[source]¶

sysinfo[source]¶

property queues: list[str] | None[source]¶

Return list of queues which are stored in queues field.

property sysinfo_json: dict | None[source]¶

add_queues(new_queues)[source]¶

Add new queue to the queues field.

remove_queues(remove_queues)[source]¶

Remove queue from queues field.

update_state(state)[source]¶

Update state field.

airflow.providers.edge3.models.edge_worker.set_metrics(worker_name, state, jobs_active, concurrency, free_concurrency, queues)[source]¶

Set metric of edge worker.

airflow.providers.edge3.models.edge_worker.reset_metrics(worker_name)[source]¶

Reset metrics of worker.

airflow.providers.edge3.models.edge_worker.get_registered_edge_hosts(states=None, session=NEW_SESSION)[source]¶

airflow.providers.edge3.models.edge_worker.request_maintenance(worker_name, maintenance_comment, session=NEW_SESSION)[source]¶

Write maintenance request to the db.

airflow.providers.edge3.models.edge_worker.exit_maintenance(worker_name, session=NEW_SESSION)[source]¶

Write maintenance exit to the db.

airflow.providers.edge3.models.edge_worker.remove_worker(worker_name, session=NEW_SESSION)[source]¶

Remove a worker that is offline or just gone from DB.

airflow.providers.edge3.models.edge_worker.change_maintenance_comment(worker_name, maintenance_comment, session=NEW_SESSION)[source]¶

Write maintenance comment in the db.

airflow.providers.edge3.models.edge_worker.request_shutdown(worker_name, session=NEW_SESSION)[source]¶

Request to shutdown the edge worker.

airflow.providers.edge3.models.edge_worker.add_worker_queues(worker_name, queues, session=NEW_SESSION)[source]¶

Add queues to an edge worker.

airflow.providers.edge3.models.edge_worker.remove_worker_queues(worker_name, queues, session=NEW_SESSION)[source]¶

Remove queues from an edge worker.