https://dagster.io/ logo
#ask-community
Title
# ask-community
r

Rhys Goodall

06/07/2023, 4:38 PM
We want to use a static partition of logically separate out a data asset into more modular units that are used by different downstream reporting functions. We’ve implemented this with a static partition but when we attempt to run a job to materialise all the backfill aspect triggers multiple runs to rematerialise the assets upstream of the partitioned asset. I guess this is due to partitioned assets originally only supporting time partitioned assets where the first asset level in DAG would all be time partitioned.
s

sandy

06/07/2023, 9:16 PM
when we attempt to run a job to materialise all the backfill aspect triggers multiple runs to rematerialise the assets upstream of the partitioned asset
Is my understanding correct that those upstream assets are non-partitioned? If you launch the backfill from the asset graph (instead of from the job page), it should only materialize each of those once. Here's an issue that tracks having the same behavior work when launching a backfill from the job page : https://github.com/dagster-io/dagster/issues/11962.
r

Rhys Goodall

06/08/2023, 5:07 PM
yes the upstream assets are not partitioned
If you launch the backfill from the asset graph (instead of from the job page), it should only materialize each of those once.
It only triggers a single job even if selecting multiple runs for the backfill which I think is correct.
3 Views