Is there a way to mark partitions as Skipped or Succeeded? • The use case here is that I want to re-partition an asset, but don’t want to have to backfill. I also don’t want partitions that previously ran to be considered “missing” • ex. Imagine I have partitions defining ranges 0-99, 100-199 …, but want to change the partitions to represent 0-9, 10-19 … Related, is there a max # of partitions that an asset could have (hopefully without causing perf regressions in Dagit)

hi @Alex Kan! There's currently no direct way to mark partitions with a given status, but the general recommendation here is just to write a dummy op that yields a bunch of AssetMaterialization events with the partition keys you want to show as "filled in" (i.e.

yield AssetMaterialization(asset_key=AssetKey("my_key"), partition="0-9")

)

Yep, I think you should be okay with 20K partitions

Ah that’s smart! Could easily write a utility script to yield those events. Was playing around earlier and bricked Dagit with 70k partitions and should def be able to get that down. Thanks all!

(also nice to meet you not outside murraccis 😄)

Was that the first Dagster celebrity sighting? 😛

@owen I assume this works only for the asset partition view. Is there a way we can mark the corresponding job partitions (for the assets) as succeeded as well?