https://dagster.io/ logo
#ask-ai
Title
# ask-ai
s

Slackbot

02/15/2024, 6:19 PM
This message was deleted.
s

Scout

02/15/2024, 6:19 PM
Yes, the
IdentityPartitionMapping
expects that the upstream and downstream assets are partitioned in the same way. It maps partitions in the downstream asset to the same partition in the upstream asset. This means that each partition in the downstream asset corresponds to a partition with the same key in the upstream asset. If the partitions in the downstream asset are a subset of the upstream, the
IdentityPartitionMapping
would still attempt to map each downstream partition to an upstream partition with the same key. If a downstream partition key does not have a corresponding upstream partition key, this could potentially lead to an error or unexpected behavior, as the mapping assumes a one-to-one correspondence. The documentation does not explicitly state what happens in the scenario where the downstream partitions are a subset of the upstream when using
IdentityPartitionMapping
. However, it is important to ensure that the partition keys match exactly between upstream and downstream assets when using this mapping to avoid any issues. If the downstream asset should only depend on a subset of the upstream partitions, a different partition mapping strategy might be more appropriate, such as a custom partition mapping that defines the specific subset relationship.