Manan P
02/01/2023, 12:44 AM@asset(
partitions_def=MultiPartitionsDefinition(
{
"customer_code": StaticPartitionsDefinition(["abc","youi"]),
"datasource_name": StaticPartitionsDefinition(["tv", "search"]),
}
)
)
So the above would create the following partitions: ['abc|tv', 'abc|search', 'youi|tv', 'youi|search'].
But this may not be the case for us, each customer can have a different list of data sources.
Is there a way to code a mapping such that is produces:
['abc|tv', 'abc|search', 'youi|tv'].
i.e. 'youi' does not have the 'search' datasource.Sean Davis
02/01/2023, 4:23 AMcustomer, datasource = partition_key.split('|')
# perform some work that depends on customer and datasource
....
This will work just fine. You'll just see the single partition key in the dagit UI, though.jamie
02/01/2023, 4:32 PM