https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Alvaro Gomez Covella

03/16/2023, 6:59 PM
Hi team wave anim, We’d like to implement a
job
that runs at daily basis that every day re-execute/materialized the last N days (fix number). We already have a Dagster pipeline that used assets (
DailyPartitionsDefinition
) up and running. We even can achieve this with
Dagit
. But for whatever reason, I’m not able to create a job (
define_asset_job()
) that backfill N days. I’ve found this but I don’t know if it is exactly what I want. Means re-write our custom IOManager. Also found this unresolved open issue that might help. But I cannot find anything related to
launchPartitionBackfill
in the docs. I’m coming from Airflow world, so the first idea that come to my mind is to have a daily normal
DAG
that handles yesterday, and a second
DAG
that “clears” the last week execution. Is this possible in Dagster? or better, What is the most elegant way in Dagster? Thanks
y

yuhan

03/16/2023, 9:21 PM
a

Alvaro Gomez Covella

03/17/2023, 9:46 AM
Thanks! Is there any python module to do it in Dagster? Or should I use the approach of executing Shell commands with Python (e.g.:
subprocess
Module)?
a

Alexander Kerckhofs

03/17/2023, 10:46 AM
a

Alvaro Gomez Covella

03/17/2023, 10:47 AM
Yeah, thanks Alexander.. I was looking at it. I may have found the right function: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster/dagster/_cli/job.py#L842-L851 I’ll keep digging. Thanks again