Olivier Dupuis
07/06/2023, 1:05 PMjohann
07/06/2023, 8:17 PMasset_2, which uses a lazy policy, is expected to update whenever asset_1 changesThis isn’t quite what I’d expect for lazy auto materialize policies-
AutoMaterilaizePolicy.eager()
would materialize as soon as the upstream changes. Lazy will wait until the freshness policy is overdue. What is your FreshnessPolicy set to?johann
07/06/2023, 8:18 PMYet, asset_2 indicates new changes in upstream data despite being materialized after asset_1This sounds like a bug, is there any other info you could share to help reproduce?
Olivier Dupuis
07/06/2023, 8:22 PMeager
. Here’s how it’s defined in that dbt model…
{{
config(
dagster_freshness_policy = {"maximum_lag_minutes": 24*60},
dagster_auto_materialize_policy = {"type": "eager"},
)
}}
In Dagit, this is how the policies are interpreted…
This asset is automatically re-materialized when at least one of the following are true:
- it is missing
- it has a freshness policy that requires more up-to-date data
- any of its descendants have a freshness policy that require more up-to-date data
- any of its parent assets / partitions have newer data
and none of the following are true:
- any of its parent assets / partitions are missing
- any of its ancestor assets / partitions have ancestors of their own with newer data
Happy to share more info if needed.johann
07/06/2023, 8:23 PMOlivier Dupuis
07/06/2023, 8:29 PMOlivier Dupuis
07/06/2023, 8:30 PMstg_protest_groupings
asset, I have the same data version…
input_data_version/airbyte/protest_groupings 40a75b5a24fd5bb9f908788374195d28c6f8aecefdb9d13e153439c00845ecb8