airflow.example_dags.plugins.custom_partition_mapper

Classes

`PrefixStripMapper`	A partition mapper that strips a fixed namespace prefix from upstream keys.
`PrefixStripMapperPlugin`	Class used to define AirflowPlugin.

Module Contents

class airflow.example_dags.plugins.custom_partition_mapper.PrefixStripMapper(prefix, *, separator='::', max_downstream_keys=None)[source]

Bases: airflow.partition_mappers.base.PartitionMapper

A partition mapper that strips a fixed namespace prefix from upstream keys.

Upstream systems often qualify partition keys with a region or environment prefix — for example "eu::daily-sales" or "us::daily-sales". A downstream asset that aggregates across regions only cares about the base key ("daily-sales"). PrefixStripMapper strips the given prefix (including a configurable separator) so that all upstream namespaces collapse to the same downstream partition key.

If the upstream key does not start with the configured prefix the key is returned unchanged, which is deliberate: keys that already live in the target namespace pass through without modification.

This class demonstrates registering a custom PartitionMapper subclass via the AirflowPlugin.partition_mappers registry. Any plugin that lists it in partition_mappers = [...] makes it available to PartitionedAssetTimetable and RollupMapper without modifying core Airflow.

Parameters:

prefix (str) – The namespace prefix to strip, e.g. "eu".
separator (str) – The string that separates the prefix from the base key. Defaults to "::" to match a common "region::key" convention.

prefix[source]

separator = '::'[source]

to_downstream(key)[source]

Return the target key that the given source partition key maps to.

serialize()[source]

classmethod deserialize(data)[source]

class airflow.example_dags.plugins.custom_partition_mapper.PrefixStripMapperPlugin[source]

Bases: airflow.plugins_manager.AirflowPlugin

Class used to define AirflowPlugin.

name = 'prefix_strip_mapper_plugin'[source]

partition_mappers[source]