John Mav

07/31/2020, 10:59 PM
Is there any way to currently trigger a partition backfill but insure that partitions run in some specified ordering? At the moment it seems like partitions will just run/finish in whatever order they decide to start.


07/31/2020, 11:43 PM
No, currently the backfill is just run by launching all of the runs.
You might be able to put in some custom logic in your run launcher that enforces some sort of priority
I believe the runs are launched in the order that the are returned by the partition generation function

John Mav

08/03/2020, 3:41 PM
gotchya, I’ll dig around some of the run launcher logic! Thank you 🙂