Robele Baker
02/20/2024, 2:21 PMget_asset_key_for_model
function.
When I define all my assets, I receive duplicate key violations.
all_assets = [
*dbt_assets,
*upstream_dbt_assets,
*downstream_dbt_assets,
]
When I Python Debug DBT, it looks like the downstream_dbt_assets
job includes all assets in dbt_assets
.
Does anyone know if this is immediately an issue with my asset selection in all_assets
or if there is something wrong with my implementation of downstream_dbt_assets
?
I can provide more colour as needed.Robele Baker
02/20/2024, 2:29 PM*dbt_assets
from the all_assets
list. However, it just feels like I'm solving a sympton.rex
02/20/2024, 2:33 PMload_assets_from_package_module
?
There might be an interaction with:
• get_asset_key_for_model
(which requires a dbt_assets
definition)
• And load_assets_from_package_module
which loads all assets from a package module
If you imported dbt_assets
in your package module that defines your downstream dbt assets, load_assets_from_package_module
will pick up both of them.Robele Baker
02/20/2024, 2:42 PMrex
02/20/2024, 2:50 PMdownstream_dbt_assets
module:
e.g.
# asset_keys.py
my_downstream_asset_key = get_asset_key_for_model([dbt_project_assets], "my_downstream_asset_key")
# downstream_dbt_assets.py
from ..asset_keys import my_downstream_asset_key
@asset(deps=[my_downstream_asset_key]
def my_downstream_asset(...):
...
Robele Baker
02/20/2024, 3:43 PMfrom ..asset_keys import my_downstream_asset_key
@asset(
ins={
"useful_dbt_output": AssetIn(
input_manager_key="bigquery_pandas_io_manager",
key=my_downstream_asset_key,
)
},