https://dagster.io/ logo
#ask-ai
Title
# ask-ai
k

Keni Mou

03/04/2024, 7:31 PM
How do I mark asset that's not materialized in the job? There are cases when I want to run the job and realize the data is not available yet, I'd like to just skip it
s

Scout

03/04/2024, 7:31 PM
To handle cases where you want to skip the materialization of an asset when the data is not available, you can use the
output_required
parameter in your asset definition. Setting
output_required
to
False
allows you to conditionally skip the materialization of an asset without marking the run as failed. Here's an example from the documentation on how you might define such an asset:
Copy code
python @asset(partitions_def=..., output_required=False) def my_asset():     if data_is_available():         yield Output()
In this example,
data_is_available()
is a placeholder for whatever logic you need to determine if the data is available. If the data is not available, the function completes without yielding an
Output
, and the asset's materialization is skipped. This approach allows you to define assets that are only materialized when certain conditions are met, such as when the necessary data is available. If the conditions are not met, the asset is simply not materialized, and the job can continue with other assets.
k

Keni Mou

03/04/2024, 7:32 PM
does
return None
work the same way?
@Scout Does
return None
work the same way?
s

Scout

03/04/2024, 7:33 PM
Based on the discussion in the GitHub issue, there seems to be some confusion around the use of
return None
to indicate that no materialization happened. In the context of conditional materialization, you indicate that no materialization happened by omitting a
yield Output
statement rather than returning
None
. This is because in the world of assets that do not use I/O managers, you typically do not return anything (
None
is the default return when nothing is explicitly returned). Here is an example from the GitHub issue that demonstrates conditional materialization:
Copy code
python @asset(output_required=False) def make_file_randomly(context):     if random.randint(1, 10) < 5:         <http://context.log.info|context.log.info>("WRITING THE FILE")         with open("foo.txt", "w") as f:             f.write("foo")             yield Output(None)
In this example, the
yield Output(None)
is used to indicate that the asset was materialized, even though the actual output is
None
. If the condition is not met, the function will complete without yielding an
Output
, and the asset will not be materialized. So, to answer your question,
return None
does not serve the same purpose as omitting
yield Output
in the context of conditional materialization. The key is whether or not the
yield Output
statement is executed to indicate materialization. If you want to skip materialization, you should not yield an
Output
.
38 Views