Gustavo Carvalho
05/19/2023, 1:00 PMrealtime_partitions_def = TimeWindowPartitionsDefinition(
cron_schedule="*/5 * * * *",
start=pd.Timestamp("now", tz=TIMEZONE).normalize().strftime(realtime_fmt),
timezone=TIMEZONE,
fmt=realtime_fmt,
)
I have some related questions:
• How often will Dagit reload this definition causing the partition start
to be updated?
• Even if I update the partition start
, I guess there will be residual data in the dagster DB regarding previous days partitions. How can I clean it up?
EDIT: Also posted as a GitHub DiscussionGustavo Carvalho
05/19/2023, 4:58 PM