Gabe Schine
02/03/2023, 7:36 PMasset
is materialized at least every hour, either through reconciliation with upstream, or through some other mechanism. What is the best mechanism to use?FreshnessPolicy
is not right, nor is an asset_sensor
Sean Lopp
02/03/2023, 8:28 PMGabe Schine
02/03/2023, 9:22 PMA -> B
where B
is downstream of A
, B
is materialized every 24h even if A
is not, because the upstream data hasn't changed.FreshnessPolicy
of 24h, B
still shows as late because its upstream dependency is older than 24hsandy
02/03/2023, 10:46 PMA
doesn't change, why do you want B
updated? Does B
depend on other data that does change?Gabe Schine
02/03/2023, 10:49 PMB
is a calendar-week filtered view of A
, so as time passes B
will change without the upstream source data changingsandy
02/03/2023, 10:53 PM