The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

is it possible to execute partitions in order when materializing an asset? My nth partition depends on the (n-1)th partition so I'm wondering if it's possible to enforce a strict order here

You can declare self dependencies using partition mappings

You could then have it materialise itself with an auto materialisation policy

Although depending on the other partitions this issue might be relevant: <https://github.com/dagster-io/dagster/issues/16890|https://github.com/dagster-io/dagster/issues/16890>

No worries. I’m not entirely sure whether or not backfills respect self partition mappings at the moment though 

Also, is this an asset that you could materialise multiple contiguous partitions at the same time? If so, I’d look into backfill policies

I don't think so unfortunately, they have to be executed one at a time