https://dagster.io/ logo
#ask-community
Title
# ask-community
c

Chaitanya Chandurkar

02/09/2023, 8:59 PM
Hey guys, how do you typically handle backfills? I have a date partitioned job that runs every 5 mins. Is there a way to backfill multiple partitions in a single run? My job consumes a lot of resources and I don't want to trigger separate job for each partition that I want to backfill. In case of failures, is there a way I can trigger a separate backfill job that runs over multiple partitions and mark the originally failed run as succeeded manually?
s

sean

02/10/2023, 2:08 PM
Hi Chaitanaya, Filling in a partition range in a one-run backfill isn’t a first-class feature yet but we’re working towards it. In the meantime check out: https://github.com/dagster-io/dagster/discussions/11653
is there a way I can trigger a separate backfill job that runs over multiple partitions and mark the originally failed run as succeeded manually?
Run status is immutable, so there’s no way to change it. I think I understand why you’d like to though-- maybe what you’ll ultimately is partitioned assets with staleness-tracking support. Then you could tell when your partitions were up-to-date. We’re currently working on this feature.
21 Views