https://dagster.io/ logo
#ask-community
Title
# ask-community
c

Chris Evans

12/22/2021, 4:12 PM
Hey guys, I think the 
<date/time-partition>_partitioned_config
 decorators are missing the following respective offsets/params  
minute_of_hour=None, hour_of_day=None, day_of_week=None, day_of_month=None
that are present on the 
build_schedule_from_partitioned_job
 helper function.  Unless I'm mistaken, without these params we get into a weird scenario where if we need an offset we have to apply it against the injected date/time partition when building the 
run_config
 fx. For example, for a weekly partitioned config the partitions seem to be generated based on this cron 
0 0 * * 0
 ; if I actually care about weekly partitions on Mondays then I would have to timedelta the injected data/time partition to create 
0 0 * * 1
.  Furthermore, we will have partitions displayed in Dagit that aren't relevant e.g. partition 
2021-05-02
 (Sunday) when I actually care about 
2021-05-03
 .
o

owen

12/22/2021, 6:36 PM
Hi @Chris Evans! For this case, could you use the
start_date
argument of
weekly_partitioned_config
to accomplish this? You could create a schedule where the first day was some Monday, and all of the deltas would be respective to that time, giving you the offset you're looking for I believe.
c

Chris Evans

12/22/2021, 6:40 PM
Thanks for the reply. The following examples are w/ wkly partition config. When I have a start date of
2021-05-03
the first partition will be
2021-05-09
(the following sunday). If I have a start
2021-05-02
(Sunday) then the first partition will be
2021-05-02
. It seems to follow this cron:
0 0 * * 0
o

owen

12/22/2021, 6:40 PM
oh interesting -- let me try that out. doesn't seem like intended behavior
ah I see -- yeah this is a bit confusing and the documentation is a bit misleading. I thought the underlying
0 0 * * 0
cron schedule would be offset using information from
start_time
but this isn't how it works at the moment.
c

Chris Evans

12/22/2021, 6:54 PM
Gotcha, yea I don't think that is the intended behavior also. I have only tested the weekly version; not sure about the others
o

owen

12/22/2021, 6:58 PM
c

Chris Evans

12/22/2021, 7:08 PM
👍 . Not sure if this applies for the hourly/daily one too
3 Views