How do I mark asset that's not materialized in the job? There are cases when I want to run the job and realize the data is not available yet, I'd like to just skip it

To handle cases where you want to skip the materialization of an asset when the data is not available, you can use the

output_required

parameter in your asset definition. Setting

output_required

to

False

allows you to conditionally skip the materialization of an asset without marking the run as failed. Here's an example from the documentation on how you might define such an asset:

python @asset(partitions_def=..., output_required=False) def my_asset():     if data_is_available():         yield Output()

In this example,

data_is_available()

is a placeholder for whatever logic you need to determine if the data is available. If the data is not available, the function completes without yielding an

Output

, and the asset's materialization is skipped. This approach allows you to define assets that are only materialized when certain conditions are met, such as when the necessary data is available. If the conditions are not met, the asset is simply not materialized, and the job can continue with other assets.

does

return None

work the same way?

Based on the discussion in the GitHub issue, there seems to be some confusion around the use of

return None

to indicate that no materialization happened. In the context of conditional materialization, you indicate that no materialization happened by omitting a

yield Output

statement rather than returning

None

. This is because in the world of assets that do not use I/O managers, you typically do not return anything (

None

is the default return when nothing is explicitly returned). Here is an example from the GitHub issue that demonstrates conditional materialization:

python @asset(output_required=False) def make_file_randomly(context):     if random.randint(1, 10) < 5:         <http://context.log.info|context.log.info>("WRITING THE FILE")         with open("foo.txt", "w") as f:             f.write("foo")             yield Output(None)

In this example, the

yield Output(None)

is used to indicate that the asset was materialized, even though the actual output is

None

. If the condition is not met, the function will complete without yielding an

Output

, and the asset will not be materialized. So, to answer your question,

return None

does not serve the same purpose as omitting

yield Output

in the context of conditional materialization. The key is whether or not the

yield Output

statement is executed to indicate materialization. If you want to skip materialization, you should not yield an

Output

.