Hi, I did a backfill run on a multipartition ( date|type ) asset A run for 2023-04-04|type1 started LATER than a run for 2023-04-05|type1 Screenshots in the thread. Why would that happen? Newer data got overwritten with later data 😬

Hi Deividas. We don't have logic to force steps to run in order by partition. In general, partitioning is a way to segment your data, so you only generate the desired "slice" of your data in a partitioned run without touching data from other partitions. So in your case, it sounds like your data asset might be mutating contents outside of the running partition. Ideally, a run for

2023-04-04|type1

would only rebuild the contents for 2023-04-04, so partitions cannot overwrite each other.

Thanks, first I configured concurrency of backfills to be 1 but that made them very slow 😄 So then I refactored data loading to be sequence independent