https://dagster.io/ logo
#dagster-serverless
Title
# dagster-serverless
j

Jacob Marcil

02/15/2023, 4:33 PM
Hello everyone, I’m trying to configure an hourly schedule that execute with 1 day delay. Is there an easy way to do that? I Found some parameters from
build_schedule_from_partition_job
but they doesn’t seem to work for hourly partition. https://docs.dagster.io/concepts/partitions-schedules-sensors/schedules
o

owen

02/15/2023, 6:12 PM
Hi @Jacob Marcil! Depending on your precise setup, one option here would be to offset your partitions definition itself (using the
end_offset
parameter), so that the partition key
2023-01-05-05:00
actually corresponds to data from
2023-01-04-05:00
. In some scenarios, this makes more sense, but in others this would be confusing. Another option would be to create a custom schedule that runs hourly and manually feeds in the desired partition:
Copy code
@schedule(cron_schedule="0 * * * *", job=my_job)
def my_schedule():
    # pseudo code
    partition_key = (current_time - "1 day").to_string()
    return my_job.run_request_for_partition(partition_key=partition_key)
j

Jacob Marcil

02/15/2023, 6:24 PM
I think this would be confusing…. The reason I want to do that I only because the data might not be available before 1 day. So what I want is to query each hour for the data with the right partition key, but with a lag of 1 day…. I think changing the partition_key will create more problem than it solves. 😞
o

owen

02/15/2023, 6:24 PM
that's fair -- i think the second option would work better for you in that case
j

Jacob Marcil

02/15/2023, 6:25 PM
Ohhh in the second option my partition key would still be the same. Niceee. Thank you I’ll try that right away 🙂
Works like a charm. Thank you very much 🙂
🌈 1
3 Views