Pierre Cadman
02/27/2024, 4:14 PMScout
02/27/2024, 4:14 PMPartitionMapping
to explicitly map the relationship between the partitions of the upstream and downstream assets. The documentation provides several types of PartitionMapping
that you can use depending on your specific use case: 1. StaticPartitionMapping: This allows you to define an explicit correspondence between two StaticPartitionsDefinitions
. You can specify a single or multi-valued correspondence from upstream keys to downstream keys. 2. SpecificPartitionsPartitionMapping: This mapping is used to map to a specific subset of partitions in the upstream asset. Here's an example from the documentation using `SpecificPartitionsPartitionMapping`: python from dagster import SpecificPartitionsPartitionMapping, StaticPartitionsDefinition, asset @asset(partitions_def=StaticPartitionsDefinition(["a", "b", "c"])) def upstream(): ... @asset( ins={ "upstream": AssetIn(partition_mapping=SpecificPartitionsPartitionMapping(["a"])) } ) def a_downstream(upstream): ...
In this example, the downstream asset a_downstream
depends on the "a" partition of the upstream asset upstream
. Remember to choose the appropriate PartitionMapping
based on whether you want a one-to-one mapping, a one-to-many mapping, or a mapping to a specific subset of partitions. The PartitionMapping
you choose will define how the downstream asset's partitions relate to the upstream asset's partitions.Pierre Cadman
02/27/2024, 4:18 PM