https://dagster.io/ logo
#ask-community
Title
# ask-community
g

Gustavo Carvalho

05/19/2023, 1:00 PM
Hi! I have assets partitioned in 5-min time windows. I only care about the materializations on the current day. As such, I defined my partition as:
Copy code
realtime_partitions_def = TimeWindowPartitionsDefinition(
    cron_schedule="*/5 * * * *",
    start=pd.Timestamp("now", tz=TIMEZONE).normalize().strftime(realtime_fmt),
    timezone=TIMEZONE,
    fmt=realtime_fmt,
)
I have some related questions: • How often will Dagit reload this definition causing the partition
start
to be updated? • Even if I update the partition
start
, I guess there will be residual data in the dagster DB regarding previous days partitions. How can I clean it up? EDIT: Also posted as a GitHub Discussion
up?
2 Views