The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Is there any way to currently trigger a partition backfill but insure that partitions run in some specified ordering? At the moment it seems like partitions will just run/finish in whatever order they decide to start.

No, currently the backfill is just run by launching all of the runs.

You might be able to put in some custom logic in your run launcher that enforces some sort of priority

I believe the runs are launched in the order that the are returned by the partition generation function

gotchya, I’ll dig around some of the run launcher logic! Thank you :slightly_smiling_face: