https://dagster.io/ logo
Title
d

Danny Steffy

03/28/2023, 5:32 PM
is there a way to have a partitioned asset re-run previous day's partitions? For instance - I have a daily partitioned asset with future data (a week forward in time) that I want to load. However, once the day is completed, the previous day's data changes and I want to to reflect that. So basically I have a daily partitioned asset that I want to schedule a job for that not only updates the previous day's data, but also get the data for the next day 7 days ahead as well.
j

Jack Yin

03/28/2023, 5:47 PM
the way i’ve handled this stuff is by fanning out the date ranges with a single day’s partition - e.g. for a single partition input X, the function upserts data from T = X - 7 to T = X
s

sean

03/28/2023, 5:56 PM
I’m not sure I completely grok this scenario but it is possible to issue multiple
RunRequest
from a schedule, so you could launch runs both the current and previous day’s partition
d

Danny Steffy

03/28/2023, 8:21 PM
I'm currently using
build_schedule_from_partitioned_job
to build my schedule, would I need to scrap that and write a custom schedule?