https://dagster.io/ logo
#ask-community
Title
# ask-community
m

Matthias Queitsch

02/27/2023, 2:24 PM
Hi, is there a way to let dagster automatically check if a SourceAsset has been refreshed? I have setup a SourceAsset by using @observable_source_asset and it works fine when clicking on the "observer" button. But how do I let dagster check this every 30mins...
Ok, IIUC having an asset which depends on a sourceasset automatically invokes the observe_fn on the upstream sourceasset, correct? If that is the case, then I have another issue: I have created an asset from a graph with a configuration using AssetDefinition.from_graph() how to I specify the upstream dependency to the SourceAsset?
s

sandy

02/27/2023, 4:43 PM
Ok, IIUC having an asset which depends on a sourceasset automatically invokes the observe_fn on the upstream sourceasset, correct?
This is actually not currently the case @sean - do you have a recommendation here?
r

Rupert Arup

03/01/2023, 12:40 PM
This is something that would be really useful. We intend to use a combination of a freshness policy and the @observable_source_asset to determine if we need to materialise our downstream assets.
s

sean

03/02/2023, 10:29 PM
Hi Matthias, It sounds like what you need is either: • The ability to run source asset observations on a Schedule • The ability to run source asset observations as part of a job with regular asset materializations, and to control which downstream asset materializations run depending on the version returned by the source asset. Unfortunately neither of these are possible at the moment, but I we are very actively working on these two things and source assets in general, so you can look out for some improvements here in the next few weeks.
m

Matthias Queitsch

03/03/2023, 11:49 AM
Thanks @sean Yes, that is exactly what I was looking for. I will keep my eyes peeled.
s

sandy

03/22/2023, 11:51 PM
As of today's release, you can now observe source assets on a schedule: https://docs.dagster.io/concepts/assets/asset-observations#observable-source-assets
❤️ 1
3 Views