Hey everyone, I am working on a flow where an asset - A creates partitions on hourly basis, and an asset B which will depend on a set of partitions of asset A, i want to configure a range of partitions of asset A while running asset B on dagit UI, is this possible?
s
sandy
12/28/2022, 5:02 PM
Is B also hourly-partitioned?
p
Pawan Kumar Meena
12/29/2022, 4:38 AM
B is not partitioned.
Pawan Kumar Meena
12/29/2022, 6:53 AM
I found something called Partition Mapping, is there any examples available?
s
sandy
12/29/2022, 4:32 PM
In the current release, if you launch a backfill that includes both B as well as N partitions of A, B will be materialized N times (which I suspect is undesirable for you?)
I'm currently working on an improvement to this for the next release (first Thursday of January) that will make it so that B only gets materialized once, after all the partitions of A.