https://dagster.io/ logo
#ask-community
Title
# ask-community
o

Olivier Dupuis

07/06/2023, 1:05 PM
Hello, I’m facing an issue related to freshness policies for historical assets. - asset_1 hasn’t been updated since April 28th, and its data version is XYZ123 - asset_2, which uses a lazy policy, is expected to update whenever asset_1 changes - The system tags show that asset_2 has the same input data version (XYZ123) as asset_1 - Yet, asset_2 indicates new changes in upstream data despite being materialized after asset_1 Could someone help me understand why this is happening? Thank you!
j

johann

07/06/2023, 8:17 PM
Not positive what’s up, but to clarify:
asset_2, which uses a lazy policy, is expected to update whenever asset_1 changes
This isn’t quite what I’d expect for lazy auto materialize policies-
AutoMaterilaizePolicy.eager()
would materialize as soon as the upstream changes. Lazy will wait until the freshness policy is overdue. What is your FreshnessPolicy set to?
Yet, asset_2 indicates new changes in upstream data despite being materialized after asset_1
This sounds like a bug, is there any other info you could share to help reproduce?
o

Olivier Dupuis

07/06/2023, 8:22 PM
Hey @johann, thanks for looking into this. Sorry, you’re right, my freshness policy was set to
eager
. Here’s how it’s defined in that dbt model…
Copy code
{{
    config(
        dagster_freshness_policy = {"maximum_lag_minutes": 24*60},
        dagster_auto_materialize_policy = {"type": "eager"},
    )
}}
In Dagit, this is how the policies are interpreted…
Copy code
This asset is automatically re-materialized when at least one of the following are true:
- it is missing
- it has a freshness policy that requires more up-to-date data
- any of its descendants have a freshness policy that require more up-to-date data
- any of its parent assets / partitions have newer data

and none of the following are true:
- any of its parent assets / partitions are missing
- any of its ancestor assets / partitions have ancestors of their own with newer data
Happy to share more info if needed.
j

johann

07/06/2023, 8:23 PM
Could you share an image of the incorrect upstream newer label? Can dm if the asset names are sensitive
o

Olivier Dupuis

07/06/2023, 8:29 PM
This is the lineage I have between asset_1 (protest_groupings) and asset_2 (stg…protest_groupings). As you can see, the first asset is an “empty” asset, as this is a dbt source pointing to Airbyte. I used to refresh its content in Dagster through a resource, but dropped it since that source table doesn’t update anymore. As you can see in the second image, the last time it materialized was April 28.
And when I look at the system tags for the
stg_protest_groupings
asset, I have the same data version…
Copy code
input_data_version/airbyte/protest_groupings	40a75b5a24fd5bb9f908788374195d28c6f8aecefdb9d13e153439c00845ecb8
3 Views