Vinnie
08/23/2022, 2:15 PMmulti_asset
that yields optional outputs; and their downstream dbt models, created with define_asset_job(selection=AssetSelection.keys().downstream())
. If not all assets from the multi_asset
are materialized by the pipeline/first op, execution of downstream ops (dbt models) doesn’t seem to run, returning a log message Skipping step <dbt_op> due to skipped dependencies: [].
Works fine and materializes everything as expected if all assets are yielded.
Anything I’m missing?multi_asset
logic
@multi_asset(
outs={
"asset1": AssetOut(
key_prefix="some_prefix",
is_required=False,
),
"asset2": AssetOut(
key_prefix="some_prefix",
is_required=False,
),
},
can_subset=True,
group_name="my_cool_group",
)
def my_assets(context):
elems = fetch()
for elem in elems:
if elem["dest_asset"] in context.selected_output_names:
processed_elem = process(elem)
yield Output(value=processed_elem["data"], output_name=elem["dest_asset"])
chris
08/24/2022, 4:59 PMVinnie
08/24/2022, 5:34 PMchris
08/24/2022, 7:18 PMload_assets_from_dbt_project
so that there are two different software artifacts constructedVinnie
08/24/2022, 7:23 PMchris
08/24/2022, 7:30 PM