What you can build¶
Each page starts from a job a data team already has, shows the Dag that does it, and names what Airflow adds over a script: a schedule, a retryable task per item, an approval gate, or a fan-out sized at runtime. Every Dag ships with the provider and runs against your connections.
Start with Triage support tickets. It needs nothing but an LLM connection and shows the shape most of the others build on: structured output plus dynamic task mapping.
Every Dag needs the provider installed with the extra for your model vendor and a
pydanticai connection named pydanticai_default; Quick start covers both.
Each page’s “Run it” lists only what that Dag adds.
Scenario |
What the model does |
What Airflow does |
|---|---|---|
Reads each ticket and returns priority, category, summary and next action as a typed record |
One retryable task per ticket, results in XCom, one argument away from a schedule |
|
Picks rerun, page or ignore from the error text, with a confidence score |
Runs only the chosen branch, sends low-confidence picks to a human |
|
Compares source and target schemas and reports what would break a load |
Branches to the load or to a notification, no migration ever runs |
|
Queries the warehouse and does the pivots in a sandbox to explain a revenue move |
Injects the date, holds the credential, destroys the sandbox when the run ends |
|
Turns a fixed question into SQL over a CSV |
Downloads the file monthly, records schema changes before generating SQL, emails the result |
|
Splits a comparison question per company and writes the report |
Weekly indexing Dag, on-demand analysis Dag, per-company fan-out, review at both ends |
|
Answers a question from retrieved excerpts, citing them |
Weekly indexing Dag, query Dag triggered with a |
|
Assesses progress per proposal, then checks its own report against the evidence |
Mapped evidence gathering, deterministic correction step, human review |
|
Labels sentiment for every review in one batch job |
Waits up to 24 hours without holding a worker, lands results on object storage |
|
Decides which tools to call to answer a research question |
Human edits the question first, separate formatting step, approval before delivery |
More ideas¶
The same operators cover many other jobs. These do not have an example Dag yet, but each can be built from the patterns shown on the pages above.
Explain a failure in the alert. A task with
trigger_rule="one_failed"reads the failed task’s log,@task.llmreturns aLiteralroot cause and a two-sentence explanation, and the notifier posts that instead of a stack trace.Data-quality triage. Feed null rates, freshness and duplicate counts from your checks task to
@task.llmwith aFindinglist asoutput_type, thenLLMBranchOperatorto page, file a ticket, or ignore.Daily incident digest. Fetch alerts for the data interval, one
@task.llmsummary per service with.expand(), one synthesis call,ApprovalOperatorbefore it posts.Release notes from merged pull requests. Weekly
HttpOperatorfetch,@task.llm_batchto classify each PR at batch prices, one call to draft the notes, aHITLEntryOperatorfor the editor’s pass.Invoices into a table.
ObjectStoragePathlists new files,@task.llm_file_analysisextracts a typed row from each with.expand(), a SQL operator inserts them, and the Dag emits an Asset so downstream reporting runs when the rows land.Rewrite a failing query. On a SQL task’s failure, hand the query and the database error to
@task.llm_sqlwith the schema context, and put the rewrite in front of a reviewer before it runs.Tag and route incoming files.
@task.llm_file_analysison each new object decides its type and sensitivity,@task.branchmoves it to the right bucket.Catalog descriptions. Nightly, for every table that changed,
@task.llmwrites a column-level description from the schema and a sample, and a task upserts it into the catalog.
Example Dags lists the same Dags by operator.
Note
Dags with HITLEntryOperator or ApprovalOperator pause under airflow dags test
until someone answers from Required Actions in the UI of an api-server on the same
metadata database. airflow standalone gives you one.