Hi, is there a way to let dagster automatically ch...
# ask-community
m
Hi, is there a way to let dagster automatically check if a SourceAsset has been refreshed? I have setup a SourceAsset by using @observable_source_asset and it works fine when clicking on the "observer" button. But how do I let dagster check this every 30mins...
Ok, IIUC having an asset which depends on a sourceasset automatically invokes the observe_fn on the upstream sourceasset, correct? If that is the case, then I have another issue: I have created an asset from a graph with a configuration using AssetDefinition.from_graph() how to I specify the upstream dependency to the SourceAsset?
s
Ok, IIUC having an asset which depends on a sourceasset automatically invokes the observe_fn on the upstream sourceasset, correct?
This is actually not currently the case @sean - do you have a recommendation here?
r
This is something that would be really useful. We intend to use a combination of a freshness policy and the @observable_source_asset to determine if we need to materialise our downstream assets.
s
Hi Matthias, It sounds like what you need is either: • The ability to run source asset observations on a Schedule • The ability to run source asset observations as part of a job with regular asset materializations, and to control which downstream asset materializations run depending on the version returned by the source asset. Unfortunately neither of these are possible at the moment, but I we are very actively working on these two things and source assets in general, so you can look out for some improvements here in the next few weeks.
m
Thanks @sean Yes, that is exactly what I was looking for. I will keep my eyes peeled.
s
As of today's release, you can now observe source assets on a schedule: https://docs.dagster.io/concepts/assets/asset-observations#observable-source-assets
❤️ 1