• Home
• Administration and Deployment

Administration and Deployment

This section contains information about deploying Dags into production and the administration of Airflow deployments.

• Production Deployment
  • Database backend
  • Multi-Node Cluster
  • Logging
  • Configuration
  • Scheduler Uptime
  • Production Container Images
  • Helm Chart for Kubernetes
  • Live-upgrading Airflow
  • Kerberos-authenticated workers
  • Secured Server and Service Access on Google Cloud
• Logging & Monitoring
  • Logging and Monitoring architecture
  • Logging for Tasks
  • Advanced logging configuration
  • Metrics Configuration
  • Metric Descriptions
  • Traces Configuration
  • Callbacks
  • Checking Airflow Health Status
  • Error Tracking
• Kubernetes
  • Helm Chart for Kubernetes
• Lineage
• Listeners
  • Lifecycle Events
  • DagRun State Change Events
  • TaskInstance State Change Events
  • Asset Events
  • Dag Import Error Events
  • Usage
  • Compatibility note
• Dag Bundles
  • Why are Dag bundles important?
  • Types of Dag bundles
  • Configuring Dag bundles
  • Writing custom Dag bundles
• Dag Serialization
  • Dag Serialization Settings
  • Limitations
  • Using a different JSON Library
  • Dag Serialization with Default Values (Airflow 3.1+)
• Modules Management
  • How package/modules loading in Python works
  • Typical structure of packages
  • Built-in PYTHONPATH entries in Airflow
  • Best practices for your code naming
  • Inspecting your PYTHONPATH loading configuration
  • Adding directories to the PYTHONPATH
  • Creating a package in Python
• Scheduler
  • Running More Than One Scheduler
  • Fine-tuning your Scheduler performance
• Dag File Processing
• Fine-tuning your Dag processor performance
  • What impacts Dag processor’s performance
  • How to approach Dag processor’s fine-tuning
  • What resources might limit Dag processors’s performance
  • What can you do, to improve Dag processor’s performance
  • Dag processor Configuration options
• Pools
  • Using multiple pool slots
• Cluster Policies
  • How do define a policy function
  • Available Policy Functions
  • Examples
• Priority Weights
  • Custom Weight Rule
• Web Stack
  • Configuration
  • Separating API Servers
• Plugins
  • What for?
  • Why build on top of Airflow?
  • Available Building Blocks
  • When are plugins (re)loaded?
  • Interface
  • Plugin Management Interface
  • External Views
  • Example
  • Exclude views from CSRF protection
  • Plugins as Python packages
  • Flask Appbuilder and Flask Blueprints in Airflow 3
  • Troubleshooting

Previous Next

Was this entry helpful?