ERD Schema of the Database

Here is the current Database schema diagram.

Warning

The ER diagram shows the snapshot of the database structure valid for Airflow version 3.4.0 and it should be treated as an internal detail. It might be changed at any time and you should not directly access the database to retrieve information from it or modify the data - you should use Airflow public REST API reference to do that instead. The main purpose of this diagram is to help with troubleshooting and understanding of the internal Airflow DB architecture in case you have any problems with the database - for example when dealing with problems with migrations. See also Reference for Database Migrations for list of detailed database migrations that are applied when running migration script and db command for the commands that you can use to manage the migrations.

erDiagram connection { INTEGER id PK VARCHAR(250) conn_id VARCHAR(500) conn_type TEXT description TEXT extra VARCHAR(500) host BOOLEAN is_encrypted BOOLEAN is_extra_encrypted TEXT login TEXT password INTEGER port VARCHAR(500) schema VARCHAR(50) team_name FK } dag_bundle_team { VARCHAR(250) dag_bundle_name PK, FK VARCHAR(50) team_name PK, FK } job_team { INTEGER job_id PK, FK VARCHAR(50) team_name PK, FK } team { VARCHAR(50) name PK } job { INTEGER id PK JSON bundle_names VARCHAR(250) dag_id TIMESTAMP end_date VARCHAR(500) executor_class VARCHAR(500) hostname VARCHAR(30) job_type TIMESTAMP latest_heartbeat CHAR(32) registration_id CHAR(32) session_id TIMESTAMP start_date VARCHAR(20) state VARCHAR(1000) unixname } connection_test_request { CHAR(32) id PK VARCHAR(250) active_connection_id BOOLEAN commit_allows_update BOOLEAN commit_on_success VARCHAR(500) conn_type VARCHAR(250) connection_id TIMESTAMP created_at VARCHAR(256) executor TEXT extra VARCHAR(500) host BOOLEAN is_encrypted BOOLEAN is_extra_encrypted TEXT login TEXT password INTEGER port VARCHAR(256) queue VARCHAR(2000) result_message VARCHAR(500) schema VARCHAR(20) state VARCHAR(50) team_name VARCHAR(64) token TIMESTAMP updated_at } dynamic_region { CHAR(32) id PK TIMESTAMP created_at VARCHAR(250) dag_id FK CHAR(32) family_root_id CHAR(32) forked_from_region_id VARCHAR(250) node_id CHAR(32) parent_region_id FK INTEGER parent_region_index BLOB path INTEGER resumes_from_index VARCHAR(250) run_id FK BLOB slot_key } dag_bundle { VARCHAR(250) name PK BOOLEAN active TIMESTAMP last_refreshed TEXT signed_url_template JSON template_params VARCHAR(200) version } callback { CHAR(32) id PK VARCHAR(250) bundle_name TIMESTAMP created_at JSON data VARCHAR(20) fetch_method TEXT output INTEGER priority_weight VARCHAR(10) state INTEGER trigger_id FK VARCHAR(20) type } slot_pool { INTEGER id PK TEXT description BOOLEAN include_deferred VARCHAR(256) pool INTEGER slots VARCHAR(50) team_name FK } asset_alias_asset { INTEGER alias_id PK, FK INTEGER asset_id PK, FK } asset_alias_asset_event { INTEGER alias_id PK, FK INTEGER event_id PK, FK } asset_watcher { INTEGER asset_id PK, FK INTEGER trigger_id PK, FK VARCHAR(1500) name } asset_alias { INTEGER id PK VARCHAR(1500) group VARCHAR(1500) name } asset { INTEGER id PK TIMESTAMP created_at JSON extra VARCHAR(1500) group VARCHAR(1500) name TIMESTAMP updated_at VARCHAR(1500) uri } asset_active { VARCHAR(1500) name PK, FK VARCHAR(1500) uri PK, FK } dag_schedule_asset_name_reference { VARCHAR(250) dag_id PK, FK VARCHAR(1500) name PK TIMESTAMP created_at } dag_schedule_asset_uri_reference { VARCHAR(250) dag_id PK, FK VARCHAR(1500) uri PK TIMESTAMP created_at } dag_schedule_asset_alias_reference { INTEGER alias_id PK, FK VARCHAR(250) dag_id PK, FK TIMESTAMP created_at TIMESTAMP updated_at } dag_schedule_asset_reference { INTEGER asset_id PK, FK VARCHAR(250) dag_id PK, FK BOOLEAN allow_global_producers JSON allow_producer_teams TIMESTAMP created_at TIMESTAMP updated_at } task_outlet_asset_reference { INTEGER asset_id PK, FK VARCHAR(250) dag_id PK, FK VARCHAR(250) task_id PK JSON allow_consumer_teams BOOLEAN allow_global_consumers TIMESTAMP created_at TIMESTAMP updated_at } task_inlet_asset_reference { INTEGER asset_id PK, FK VARCHAR(250) dag_id PK, FK VARCHAR(250) task_id PK TIMESTAMP created_at TIMESTAMP updated_at } asset_dag_run_queue { INTEGER asset_event_id PK, FK VARCHAR(250) target_dag_id PK, FK INTEGER asset_id FK TIMESTAMP created_at } dagrun_asset_event { INTEGER dag_run_id PK, FK INTEGER event_id PK, FK } asset_event { INTEGER id PK INTEGER asset_id JSON extra VARCHAR(250) partition_key VARCHAR(250) source_dag_id INTEGER source_map_index VARCHAR(250) source_run_id VARCHAR(250) source_task_id CHAR(32) source_task_instance_id TIMESTAMP timestamp } asset_partition_dag_run { INTEGER id PK TIMESTAMP created_at INTEGER created_dag_run_id FK TIMESTAMP partition_date VARCHAR(250) partition_key JSON rollup_fingerprint VARCHAR(250) target_dag_id TIMESTAMP updated_at } partitioned_asset_key_log { INTEGER id PK INTEGER asset_event_id INTEGER asset_id INTEGER asset_partition_dag_run_id TIMESTAMP created_at VARCHAR(250) source_partition_key VARCHAR(250) target_dag_id VARCHAR(250) target_partition_key } deadline { CHAR(32) id PK CHAR(32) callback_id FK TIMESTAMP created_at INTEGER dagrun_id FK CHAR(32) deadline_alert_id FK TIMESTAMP deadline_time TIMESTAMP last_updated_at BOOLEAN missed } log { INTEGER id PK VARCHAR(250) dag_id TIMESTAMP dttm VARCHAR(60) event TEXT extra TIMESTAMP logical_date INTEGER map_index VARCHAR(500) owner VARCHAR(500) owner_display_name VARCHAR(250) run_id VARCHAR(250) task_id CHAR(32) task_instance_id VARCHAR(50) team_name INTEGER try_number } backfill { INTEGER id PK TIMESTAMP completed_at TIMESTAMP created_at VARCHAR(250) dag_id JSON dag_run_conf TIMESTAMP from_date BOOLEAN is_paused INTEGER max_active_runs VARCHAR(250) reprocess_behavior TIMESTAMP to_date VARCHAR(512) triggering_user_name TIMESTAMP updated_at } backfill_dag_run { INTEGER id PK INTEGER backfill_id FK INTEGER dag_run_id FK VARCHAR(250) exception_reason TIMESTAMP logical_date VARCHAR(250) partition_key INTEGER sort_ordinal } variable { INTEGER id PK TEXT description BOOLEAN is_encrypted VARCHAR(250) key VARCHAR(50) team_name FK TEXT val } deadline_alert { CHAR(32) id PK JSON callback_def TIMESTAMP created_at TEXT description JSON interval VARCHAR(250) name JSON reference CHAR(32) serialized_dag_id FK } dag_version { CHAR(32) id PK VARCHAR(250) bundle_name VARCHAR(250) bundle_version TIMESTAMP created_at VARCHAR(250) dag_id FK TIMESTAMP last_updated JSON version_data INTEGER version_number } dag_priority_parsing_request { VARCHAR(32) id PK VARCHAR(250) bundle_name VARCHAR(2000) relative_fileloc } hitl_detail { CHAR(32) ti_id PK, FK JSON assignees TEXT body JSON chosen_options TIMESTAMP created_at JSON defaults BOOLEAN multiple JSON options JSON params JSON params_input TIMESTAMP responded_at JSON responded_by TEXT subject } task_reschedule { INTEGER id PK INTEGER duration TIMESTAMP end_date TIMESTAMP reschedule_date TIMESTAMP start_date CHAR(32) ti_id FK } xcom_v1 { INTEGER dag_run_id PK VARCHAR(512) key PK INTEGER map_index PK, FK VARCHAR(250) task_id PK, FK VARCHAR(250) dag_id FK BOOLEAN dag_result INTEGER mapped_length VARCHAR(250) run_id FK TIMESTAMP timestamp JSON value } xcom_v2 { CHAR(32) id PK BOOLEAN dag_result VARCHAR(512) key INTEGER mapped_length CHAR(32) task_instance_id FK TIMESTAMP timestamp JSON value } legacy_task_data_owner { VARCHAR(250) dag_id PK INTEGER map_index PK VARCHAR(250) run_id PK VARCHAR(250) task_id PK CHAR(32) task_instance_id FK } task_instance { CHAR(32) id PK VARCHAR(50) archived_reason JSON context_carrier VARCHAR(1000) custom_operator_name VARCHAR(250) dag_id FK CHAR(32) dag_version_id FK FLOAT duration TIMESTAMP end_date VARCHAR(1000) executor BLOB executor_config TEXT external_executor_id VARCHAR(1000) hostname TIMESTAMP last_heartbeat_at INTEGER max_tries JSON next_kwargs VARCHAR(1000) next_method VARCHAR(1000) operator INTEGER pid VARCHAR(256) pool INTEGER pool_slots INTEGER priority_weight VARCHAR(256) queue INTEGER queued_by_job_id TIMESTAMP queued_dttm CHAR(32) region_id INTEGER region_index VARCHAR(250) rendered_map_index FLOAT retry_delay_override VARCHAR(500) retry_reason VARCHAR(250) run_id FK TIMESTAMP scheduled_dttm TIMESTAMP start_date VARCHAR(20) state VARCHAR(2000) task_display_name VARCHAR(250) task_id INTEGER trigger_id FK TIMESTAMP trigger_timeout INTEGER try_number VARCHAR(1000) unixname TIMESTAMP updated_at BOOLEAN working_set } task_instance_note { CHAR(32) ti_id PK, FK VARCHAR(1000) content TIMESTAMP created_at TIMESTAMP updated_at VARCHAR(128) user_id } log_template { INTEGER id PK TIMESTAMP created_at TEXT elasticsearch_id TEXT filename } dag_run { INTEGER id PK INTEGER backfill_id FK VARCHAR(250) bundle_version INTEGER clear_number JSON conf JSON context_carrier TIMESTAMP created_at CHAR(32) created_dag_version_id FK INTEGER creating_job_id VARCHAR(250) dag_id TIMESTAMP data_interval_end TIMESTAMP data_interval_start TIMESTAMP end_date TIMESTAMP last_scheduling_decision INTEGER log_template_id FK TIMESTAMP logical_date TIMESTAMP partition_date VARCHAR(250) partition_key TIMESTAMP queued_at TIMESTAMP run_after VARCHAR(250) run_id VARCHAR(50) run_type INTEGER scheduled_by_job_id TIMESTAMP start_date VARCHAR(50) state VARCHAR(50) triggered_by VARCHAR(512) triggering_user_name TIMESTAMP updated_at } dag_run_note { INTEGER dag_run_id PK, FK VARCHAR(1000) content TIMESTAMP created_at TIMESTAMP updated_at VARCHAR(128) user_id } dag_tag { VARCHAR(250) dag_id PK, FK VARCHAR(100) name PK } dag_owner_attributes { VARCHAR(250) dag_id PK, FK VARCHAR(500) owner PK VARCHAR(500) link } dag { VARCHAR(250) dag_id PK JSON allowed_run_types JSON asset_expression VARCHAR(250) bundle_name FK VARCHAR(200) bundle_version VARCHAR(2000) dag_display_name JSON deadline TEXT description BOOLEAN exceeds_max_non_backfill BOOLEAN fail_fast VARCHAR(2000) fileloc BOOLEAN has_import_errors BOOLEAN has_task_concurrency_limits BOOLEAN is_draining BOOLEAN is_paused BOOLEAN is_stale TIMESTAMP last_expired FLOAT last_parse_duration TIMESTAMP last_parsed_time INTEGER max_active_runs INTEGER max_active_tasks INTEGER max_consecutive_failed_dag_runs TIMESTAMP next_dagrun TIMESTAMP next_dagrun_create_after TIMESTAMP next_dagrun_data_interval_end TIMESTAMP next_dagrun_data_interval_start TIMESTAMP next_dagrun_partition_date VARCHAR(255) next_dagrun_partition_key VARCHAR(2000) owners JSON partition_mapper_info VARCHAR(2000) relative_fileloc BOOLEAN timetable_asset_gated VARCHAR(1000) timetable_description BOOLEAN timetable_partitioned BOOLEAN timetable_periodic TEXT timetable_summary VARCHAR(255) timetable_type } dag_warning { VARCHAR(250) dag_id PK, FK VARCHAR(50) warning_type PK TEXT message TIMESTAMP timestamp } rtif_v1 { VARCHAR(250) dag_id PK, FK INTEGER map_index PK, FK VARCHAR(250) run_id PK, FK VARCHAR(250) task_id PK, FK JSON k8s_pod_yaml JSON rendered_fields } rtif_v2 { CHAR(32) id PK JSON k8s_pod_yaml JSON rendered_fields CHAR(32) task_instance_id FK } trigger { INTEGER id PK VARCHAR(1000) classpath TIMESTAMP created_date TEXT kwargs VARCHAR(256) queue VARCHAR(50) team_name FK INTEGER triggerer_id } asset_state_store { INTEGER asset_id PK, FK VARCHAR(512) key PK VARCHAR(250) last_updated_by_dag_id VARCHAR(16) last_updated_by_kind INTEGER last_updated_by_map_index CHAR(32) last_updated_by_region_id INTEGER last_updated_by_region_index VARCHAR(250) last_updated_by_run_id VARCHAR(250) last_updated_by_task_id CHAR(32) last_updated_by_task_instance_id INTEGER last_updated_by_try_number TIMESTAMP updated_at TEXT value } dag_favorite { VARCHAR(250) dag_id PK, FK VARCHAR(250) user_id PK } import_error { INTEGER id PK VARCHAR(250) bundle_name VARCHAR(1024) filename VARCHAR(2000) source_reference TEXT stacktrace TIMESTAMP timestamp } revoked_token { VARCHAR(255) jti PK TIMESTAMP exp } dag_code { CHAR(32) id PK TIMESTAMP created_at VARCHAR(250) dag_id CHAR(32) dag_version_id FK VARCHAR(2000) fileloc VARCHAR(64) language TIMESTAMP last_updated TEXT source_code VARCHAR(32) source_code_hash } serialized_dag { CHAR(32) id PK TIMESTAMP created_at VARCHAR(32) dag_hash VARCHAR(250) dag_id CHAR(32) dag_version_id FK JSON data BLOB data_compressed TIMESTAMP last_updated } task_state_store { INTEGER id PK VARCHAR(250) dag_id INTEGER dag_run_id FK TIMESTAMP expires_at VARCHAR(512) key CHAR(32) region_id INTEGER region_index VARCHAR(250) run_id VARCHAR(250) task_id TIMESTAMP updated_at TEXT value } team one or zero--0+ connection : has team 1--0+ dag_bundle_team : has dag_bundle 1--0+ dag_bundle_team : has team 1--0+ job_team : has job 1--0+ job_team : has dag_run 1--0+ dynamic_region : has dynamic_region one or zero--0+ dynamic_region : has dag_run 1--0+ dynamic_region : has trigger one or zero--0+ callback : has team one or zero--0+ slot_pool : has asset 1--0+ asset_alias_asset : has asset_alias 1--0+ asset_alias_asset : has asset_event 1--0+ asset_alias_asset_event : has asset_alias 1--0+ asset_alias_asset_event : has trigger 1--0+ asset_watcher : has asset 1--0+ asset_watcher : has asset 1--1 asset_active : has asset 1--1 asset_active : has dag 1--0+ dag_schedule_asset_name_reference : has dag 1--0+ dag_schedule_asset_uri_reference : has asset_alias 1--0+ dag_schedule_asset_alias_reference : has dag 1--0+ dag_schedule_asset_alias_reference : has dag 1--0+ dag_schedule_asset_reference : has asset 1--0+ dag_schedule_asset_reference : has dag 1--0+ task_outlet_asset_reference : has asset 1--0+ task_outlet_asset_reference : has dag 1--0+ task_inlet_asset_reference : has asset 1--0+ task_inlet_asset_reference : has asset_event 1--0+ asset_dag_run_queue : has dag 1--0+ asset_dag_run_queue : has asset 1--0+ asset_dag_run_queue : has dag_run 1--0+ dagrun_asset_event : has asset_event 1--0+ dagrun_asset_event : has dag_run one or zero--0+ asset_partition_dag_run : has callback 1--0+ deadline : has deadline_alert one or zero--0+ deadline : has dag_run one or zero--0+ deadline : has dag_run one or zero--0+ backfill_dag_run : has backfill 1--0+ backfill_dag_run : has team one or zero--0+ variable : has serialized_dag 1--0+ deadline_alert : has dag 1--0+ dag_version : has task_instance 1--1 hitl_detail : has task_instance 1--0+ task_reschedule : has legacy_task_data_owner 1--0+ xcom_v1 : has legacy_task_data_owner 1--0+ xcom_v1 : has legacy_task_data_owner 1--0+ xcom_v1 : has legacy_task_data_owner 1--0+ xcom_v1 : has task_instance 1--0+ xcom_v2 : has task_instance 1--0+ legacy_task_data_owner : has dag_run 1--0+ task_instance : has dag_run 1--0+ task_instance : has dag_version one or zero--0+ task_instance : has trigger one or zero--0+ task_instance : has task_instance 1--1 task_instance_note : has backfill one or zero--0+ dag_run : has log_template one or zero--0+ dag_run : has dag_version one or zero--0+ dag_run : has dag_run 1--1 dag_run_note : has dag 1--0+ dag_tag : has dag 1--0+ dag_owner_attributes : has dag_bundle 1--0+ dag : has dag 1--0+ dag_warning : has legacy_task_data_owner 1--0+ rtif_v1 : has legacy_task_data_owner 1--0+ rtif_v1 : has legacy_task_data_owner 1--0+ rtif_v1 : has legacy_task_data_owner 1--0+ rtif_v1 : has task_instance 1--0+ rtif_v2 : has team one or zero--0+ trigger : has asset 1--0+ asset_state_store : has dag 1--0+ dag_favorite : has dag_version 1--1 dag_code : has dag_version 1--1 serialized_dag : has dag_run 1--0+ task_state_store : has

Was this entry helpful?