airflow.providers.edge.models.edge_logs

Module Contents

Classes

EdgeLogsModel

Temporary collected logs from a Edge Worker while job runs on remote site.
class airflow.providers.edge.models.edge_logs.EdgeLogsModel(dag_id, task_id, run_id, map_index, try_number, log_chunk_time, log_chunk_data)[source]

Bases: airflow.models.base.Base, airflow.utils.log.logging_mixin.LoggingMixin

Temporary collected logs from a Edge Worker while job runs on remote site.

As the Edge Worker in most cases has a local file system and the web UI no access to read files from remote site, Edge Workers will send incremental chunks of logs of running jobs to the central site. As log storage backends in most cloud cases can not append logs, the table is used as buffer to receive. Upon task completion logs can be flushed to task log handler.

Log data therefore is collected in chunks and is only temporary.

__tablename__ = 'edge_logs'[source]
dag_id[source]
task_id[source]
run_id[source]
map_index[source]
try_number[source]
log_chunk_time[source]
log_chunk_data[source]

Was this entry helpful?