https://dagster.io/ logo
#dagster-cloud
Title
# dagster-cloud
g

geoHeil

04/29/2023, 10:46 AM
One question for freshness policies
By 12:01 GMT+2, this asset should incorporate all data up to 24 hours before that time.
I get this overdue warning - but the asset was materialized successfully approx 11:31 GMT+2.
freshness_cron = "1 12 * * *" freshness_tz = "Europe/Vienna" fp = FreshnessPolicy( maximum_lag_minutes=24 * 60, cron_schedule=freshness_cron, cron_schedule_timezone=freshness_tz, )
this is the related freshness policy
o

owen

05/01/2023, 5:10 PM
hi @geoHeil! is this asset downstream of any other assets / source assets? if so, when were those assets materialized?
g

geoHeil

05/01/2023, 8:42 PM
It is the source itself
an example/screenshot
o

owen

05/01/2023, 10:36 PM
ah I see, thanks for the screenshot! for time-partitioned assets, the freshness policy is making a statement about the state of the entire asset, i.e. "incorporate all data up to X time" means "all partitions up to X time should be filled"
g

geoHeil

05/02/2023, 3:43 AM
I see. The feature request in dagit to mark partitions which are available outside dagseter as available (or no longer sourcable) will then be needed
o

owen

05/02/2023, 6:38 PM
gotcha -- there's not a super clean way of doing this, but it is possible to manually emit those materialization events to tell Dagster that they exist: https://github.com/dagster-io/dagster/discussions/12561