https://dagster.io/ logo
#dagster-feedback
Title
# dagster-feedback
s

Stephen Bailey

05/16/2022, 5:48 PM
i'm planning on launching several jobs using hourly partitions. one thing i noticed is that if you attempt to re-fill a partition, all of the job nodes aren't cleared from the first run. 1. run partitioned job. everything compeltes successfully (all green) 2. re-run partitioned job, it fails. (one node red, rest are green) i believe in airflow, re-triggering a partition is basically the same as marking all of those tagged runs as "cleared". not sure if there's a simple way to do this in dagster, but it makes for a nicer/more comprehensible visual in my opinion.
d

dish

05/16/2022, 6:49 PM
cc @prha
p

prha

05/16/2022, 7:32 PM
Hi Stephen. Just want to confirm what is confusing here: Some of the steps are retaining their “success” status from previous runs, when really you’d want to see the entire column just contain the status from the last run?
s

Stephen Bailey

05/17/2022, 12:45 AM
that's right. my intuition is that at any given time, a partition should only have state from a single run (the latest). i don't expect this is a common problem in practice, because people wont be randomly kicking off a bunch of new backfills most of the time, but just wanted to share
👍 1
p

prha

05/17/2022, 3:42 PM
Yeah, that matches my intuition too… It’s a bug that it smushes them together.
@Dagster Bot issue Partition step matrix combines the per-step status across multiple runs
d

Dagster Bot

05/17/2022, 3:45 PM
2 Views