https://dagster.io/ logo
Title
p

Phil Whitby

11/22/2022, 5:44 PM
Hi All, I was interested to try out the freshness capability, but noticed that this wasn’t appying to DBT-generated assets, adding the following into the AssetsDefinition function call in _dbt_nodes_to_assets_ seemed to get this working properly, is this on anyone’s radar currently or is it a case of making a git pr, making a general use case rather than hardcoding may be more tricky?
freshness_policies_by_key={
    asset_key: FreshnessPolicy(maximum_lag_minutes=45) for asset_key, (input_name, _) in (asset_ins | asset_outs).items()
}
👍 1
s

sandy

11/22/2022, 5:48 PM
@owen
o

owen

11/22/2022, 5:56 PM
hi @Phil Whitby! Definitely on the radar -- I think in general we should support a post-hoc way to add FreshnessPolicies to an already-created AssetsDefinition object, so something like:
my_dbt_assets = with_freshness_policies(
    load_assets_from_dbt...(...),
    freshness_policy=FreshnessPolicy(maximum_lag_minutes=45),
    # or...
    freshness_policies_by_key={AssetKey("a"): FreshnessPolicy(...), ...}
)
additionally, it'd be neat for there to be some config you could add to the dbt project / dbt models themselves that we could parse to figure out a freshness policy. so I think in the short term, having your own override to the default behavior of
dbt_nodes_to_assets
is a reasonable solution, and we'll prioritize getting a general-purpose solution for adding freshness policies to assets for the near future
🎯 1
p

Phil Whitby

11/22/2022, 6:00 PM
Thanks for the quick response! Adding this to the assetsdefinition seems ok but I think I prefer to do this on instantiation for our use case, will keep an eye out for this for sure 🙂