https://dagster.io/ logo
#ask-ai
Title
# ask-ai
s

Slackbot

02/24/2024, 9:50 AM
This message was deleted.
s

Scout

02/24/2024, 9:50 AM
According to the documentation, source asset observations can be run as part of an asset job, and this allows you to run source asset observations on a schedule. However, it is important to note that currently, source asset observations cannot be run as part of a standard asset job that materializes assets. The
selection
argument to
define_asset_job
must target only observable source assets—an error will be thrown if a mix of regular assets and observable source assets is selected. Here is an example from the documentation on how to define an observable source asset and schedule it:
Copy code
python from dagster import (     DataVersion,     ScheduleDefinition,     define_asset_job,     observable_source_asset, )  @observable_source_asset def foo_source_asset():     content = read_some_file()     hash_sig = sha256()     hash_sig.update(bytearray(content, "utf8"))     return DataVersion(hash_sig.hexdigest())  observation_job = define_asset_job("observation_job", [foo_source_asset])  # schedule that will run the observation on foo_source_asset every day observation_schedule = ScheduleDefinition(     name="observation_schedule",     cron_schedule="@daily",     job=observation_job, )
If you want to prevent an IO manager from triggering an asset observation on source assets, you should ensure that your source assets are defined using the
@observable_source_asset
decorator and that they are not included in asset jobs that are meant to materialize assets. This way, the IO manager will not be invoked for materialization purposes on these source assets. If you encounter issues where an IO manager is unexpectedly invoked for source assets, this might be a bug or a misconfiguration. In such cases, you should review your asset definitions and job configurations to ensure they align with the guidelines provided in the documentation. If the problem persists, you may want to look for similar issues in the Dagster GitHub repository or seek further assistance from the Dagster community.
2 Views