airflow.providers.common.ai.batch.anthropic¶
Anthropic batch adapter (Message Batches API).
Talks to the anthropic SDK directly, for the same reason batch/openai.py
does. The SDK import is deferred to _build_client().
The SDK’s messages.batches.create has no batch-level metadata
parameter, so the idempotency key lives only in each request’s custom_id
prefix. That is enough to recognize results, but not to find an orphaned
batch: see AnthropicBatchAdapter.find_orphaned_batch().
Attributes¶
Classes¶
Batch adapter for Anthropic's Message Batches API. |
Module Contents¶
- class airflow.providers.common.ai.batch.anthropic.AnthropicBatchAdapter(*, api_key=None, base_url=None, client=None)[source]¶
Bases:
airflow.providers.common.ai.batch.base.BatchAdapterBatch adapter for Anthropic’s Message Batches API.
- Parameters:
api_key (str | None) – Passed straight to the
anthropic.Anthropicclient.Nonefalls back to the SDK’s own env-var resolution (ANTHROPIC_API_KEY).base_url (str | None) – Passed straight to the
anthropic.Anthropicclient.client (Any | None) – Inject a pre-built client (or a test double) instead of constructing one from
api_key/base_url.
- build_output_directive(spec)[source]¶
Translate
spec.json_schemainto this provider’s structured-output request fields.Returns
{}whenspec.is_structuredisFalse. The result is merged into every request’s body/params by the adapter itself.
- validate_requests(requests, *, model, output_spec, idempotency_key, system_prompt='', max_tokens=1024, request_params=None, **kwargs)[source]¶
Reject an over-limit or otherwise invalid batch before any network call.
Must check
len(requests)againstmax_requestsand the serialized request size (including the per-request output directive built fromoutput_spec, which a large schema repeats in every request) againstmax_payload_bytes, and callcheck_custom_id_length(). Raise a subclass ofLLMBatchInputErrornaming the limit, the actual count/size, and the.expand()remedy.**kwargscarries the same batch-level request-building context assubmit()(system_prompt,max_tokens,request_params) so the serialized-size estimate matches whatsubmitsends.
- submit(requests, *, model, idempotency_key, input_fingerprint, output_spec, system_prompt='', max_tokens=1024, request_params=None, **kwargs)[source]¶
Upload/submit the batch and return the provider’s batch id.
Where the provider offers batch-level metadata (OpenAI), record both
idempotency_keyandinput_fingerprintthere sofind_orphaned_batch()can read them back. The key alone identifies the task instance, not this submission of it: it is stable across acleareven when the prompts changed.
- get_batch(batch_id)[source]¶
Return the current status of a submitted batch. Synchronous; the trigger wraps it in
to_thread.
- find_orphaned_batch(idempotency_key, input_fingerprint, not_before)[source]¶
Return
Noneunconditionally: this adapter does not recover orphaned batches.The SDK’s batch-create call has no
metadataparameter and the list endpoint cannot filter bycustom_id. A batch could in principle be identified by listing recent batches and reading thecustom_idprefix of an ended batch’s first result, but that scan costs a result download per candidate and cannot see a batch that is still running, so it is not implemented. A crash between “submit sent” and “response recorded” therefore falls through toon_orphaned_intent.