Hi! I'm a newbie of Dagster and trying to learn how to use it. I'm writing a message to ask you a question concerning about Partitions and Schedules. I would like to make partitions that ended on the current date. Each of partitions must be scheduled to be executed on the next day. For example, in the case of that the start date is Jun 1 and today is Jun 8, the below scheduled partitions are required. • Jun 1, executed at 09:00, Jun 2 • Jun 2, executed at 09:00, Jun 3 • ... • Jun 8 (today), scheduled to be executed at 09:00, Jun 9 (next day) (totally eight partitions on Jun 8) What I have tried: (1)

dagster.DailyPartitionsDefinition(
    start_date="2023-06-01",
    hour_offset=9,
    end_offset=0,
)

It was scheduled correctly but I received seven partitions that ended on Jun 7. (2)

dagster.DailyPartitionsDefinition(
    start_date="2023-06-01",
    hour_offset=9,
    end_offset=1,
)

The eight partitions are generated but the execution schedule was on the partition key date.

As far as I know (someone correct me if I'm wrong), Dagster will run on a "Today - 1" basis. Meaning that if I run a job on midnight on the 8th of June, the start of that partition is the 7th and the end is the 8th. I think if you add "end_offset"=1 you will see the 8th partition but it will still run on the 9th?

Yes, that was what I expected. But, when I set the end_offset one, the 8th partition will run on the 8th.

Nice! Good to know it can work this way too 🙂

No... 😭 That wasn't what I wanted to do. The 8th partition should run on the 9th.

Oh then why do you want to see the 8th partition already? You can simply leave the end_offset at 0 and tomorrow it will run the partition for the 8th 🙂

Sometimes I need to see the preliminary result 😅

ahh I see. My way of doing this (and this is not necessarily the right / best way) is to have a separate job which I can run manually and pass dates through the config schema. Otherwise, you can use the "Launchpad" to launch the job without a partition and then write code in the

op

so that you get the data for "Today" if it is not a partitioned run 🙂

Many assets belong to the job. I may have to rewrite the code a lot. But if there is no better option, I might go for it. Thank you for your time! 🙏

Sure thing 🙂 I don't know how you wrote the code but I'd recommend writing each

asset

as an

op

if you haven't done so already, it makes the code less complex and entangled 🙂

I think you should be able to accomplish this with just partitioned assets, no need to use ops and config you'll need to write your own schedule function: https://github.com/dagster-io/dagster/issues/14712

It's great! That is exactly what I was looking for. Thank you @sandy.