Hey all, I don't think I understand how Freshness ...
# ask-community
Hey all, I don't think I understand how Freshness Policies from upstream assest affect downstream assets (or vice versa). If there are two assets, A -> B, and A has a freshness policy of 10 minutes and B has a freshness policy of 1 minute, I would expect that A would run every 10 minutes and B would follow a minute after A gets loaded. Instead, I'm seeing that B will cause A to run every minute. So my question, how do I get the behaviour I expect?
The use case I'm thinking is this: Asset C depends on A and B. A takes a long time to run, so we only have it run once a week overnight and never ever want it to accidentally run again during the week. B runs every hour. C just needs to run once a day to load data for a dashboard by some time. How do I set this up? I though freshness policies would be the way, but judging from my test above, this would trigger the once-a-week asset to run every day
For reference, here's my test code:
Copy code
def lag_1():
    return 1

def lag_5():
    return 1

def lag_3(lag_1, lag_5):
    return lag_1 + lag_5
And I see that lag_3 triggers lag_5 to run every 3 minutes
are you using the reconciliation sensor? If I understand correctly, the default behavior for that sensor is upstream updates causes downstream updates to follow. freshness policies town down that behavior to "it's fine to wait X minutes before I respond to an upstream materialization" in other words, I believe it's more useful when your upstream updates are more frequent than downstream. you seem to have the inverse case, upstream is "slower" and downstream is faster/more-frequent
Yeah, I have a sensor on all assets in this case
i think I came across this too when experimenting with reconciliation sensors (which I currently don't use in production)
but I believe the "correct" usage is to select downstream only, otherwise they all end up being updated on the faster schedule
Ohh, I see
And then put the upstream assets on a plain schedule, I suppose
the reconciliation-monitored assets will respond to upstream updates, which should be scheduled normally (unless they also depend on other upstreams)
as I understand it, freshness is basically "i dont need this asset to update as much as their upstreams"
Ah, yeah I'm still wrapping my head around some of the finer details of freshness policies
Thanks for your help!
but since your upstream is "slower", your downstream reconciliated assets will end up being updated weekly too, only with a couple of minutes delay