Hello! I have two dynamic partitions that are hierarchical like the following: Parent Partition keys:

A, B, C

Child Partition keys:

A_1, A_2, A_3, A_4, B_1, B_2, C_1, C_2, C_3

I’d like to have a downstream asset with a Parent Partition be able to depend on an upstream asset with a Child Partition. The intent is to have each Parent combine the data of its children. Something like this:

@asset(partitions_def=child_partition)
def child_data(context):
    parent_id, child_id = context.partition_key.split("_")  # e.g. "A_1" → ("A", "1")
    # …

@asset(
    partitions_def=parent_partition,
    ins={
        "child_data": AssetIn(
            partition_mapping=CustomPartitionMapping(
                get_downstream_key_from_upstream_key=lambda key : key.split("_")[0]  # e.g. "A_1" → "A"
            ),
        )
    },
)
def parent_data(context, child_data):
    parent_id = context.partition_key
    children = child_data.values()
    # …

I made up the

CustomPartitionMapping

class, so the above won’t work 😅. Is there a good strategy for going about this? Unfortunately, the partition keys are dynamic because they aren’t known ahead of time. After looking through the partition mapping classes, it seems like using dynamic partition keys makes this a challenge. Any help is very much appreciated, Thanks!

Hi Dennis. Do your child partition keys contain every unique combination of parent partition keys and [1, 2, 3, 4...]? If so, you could represent the child partitions def as a

MultiPartitionsDefinition

with two dynamic dimensions ([a, b, c] and [1, 2, 3, 4...])

Thanks, I’ll try that out. There will be lots of combinations that don’t exist because parents have a varying number of children.