Sandeep Mullangi
02/09/2024, 7:08 AMauto_materialize_policy = AutoMaterializePolicy.eager(
max_materializations_per_minute=100
).with_rules(
AutoMaterializeRule.skip_on_not_all_parents_updated(require_update_for_all_upstream_partitions=True)
).without_rules(
AutoMaterializeRule.skip_on_parent_missing(),
AutoMaterializeRule.skip_on_parent_outdated()
)
I have freshness policy as backup so that my downstream triggers atleast after time specified and it is also not working. i have following freshness policy
freshness_policy = FreshnessPolicy(maximum_lag_minutes=1)
My downstream model says overdue
I think precedence is given to materialization policy (1 of the skip condition is met) and then freshness is not triggered because of skip condition is met. In this case then what is the use of freshness policy if it is following materialization strictly?Sandeep Mullangi
02/09/2024, 1:51 PMSandeep Mullangi
02/09/2024, 5:25 PM