I have a question about how I can set up some declarative scheduling that archives the following:
Lets say I have a date partitioned software defined asset 'A'. The asset has a dependency on an upstream software defined asset and also the previous dates partition of itself.
We decide we want to backfill the upstream assets from date 0 to date 5. The partitions for asset 'A' from date 0-5 are now stale. However, I dont want them to all start running at the same time since if date 3 starts before date 2 finishes it will need to run again once date 2 finishes. How could I make it so that date 3 does not start until date 2 is no longer stale.
After doing a bit more digging it seems that if a dependency is not reconciled the job will not start so it should just work the way I want it to out of the box.