Vinnie
04/13/2023, 1:12 PMmulti_asset
that’s supposed to process a few csv files. 2/4 of these files are present in both partitions, plus one specific for each. Simplified, it looks like the following:
@multi_asset(
outs={
"some_asset": AssetOut(), # should include both a and b as static partitions and all dynamic
"some_other_asset": AssetOut(), # should include both a and b as static partitions and all dynamic
"a_asset": AssetOut(is_required=False), # should either only include dynamic partitions, or only "a" and dynamic
"b_asset": AssetOut(is_required=False), # should either only include dynamic partitions, or only "b" and dynamic
},
partitions_def=MultiPartitionsDefinition(
{
"static": StaticPartitionsDefinition(["a", "b"]),
"dynamic": DynamicPartitionsDefinition(name="my_dynamic_partition"),
}
),
can_subset=True,
)
currently the workaround is to just return None
if the asset is not expected to exist for the static partition called, but it would be great for observability if dagster could know that some of the assets need to be partitioned differentlyclaire
04/13/2023, 7:42 PMVinnie
04/13/2023, 8:30 PM