Romain Robert
05/10/2023, 4:23 PMload_assets_from_dbt_project
to some downstream assets (python assets).
Is there a way to do so without setting the ins
or non_argument_deps
property of our python asset to a specific asset of our dbt project ? (I am referencing the dbt downstream doc here). The doc is presenting a really simple use case where a dbt project has only one leaf in its lineage. Also, we don’t want to have to maintain Dagster each time a new asset is being added from our dbt project.
Are we trying to do something wrong ? Also, we would like to avoid using Sensors here.
dbt_assets = load_assets_from_dbt_project(
project_dir=utils.DBT_PROJECT_PATH,
profiles_dir=utils.DBT_PROFILES,
use_build_command=True,
key_prefix="dbt",
node_info_to_group_fn=lambda k: "group1"
)
@asset(
non_argument_deps=[AssetKey(["dbt", "*"])], # All my dbt assets <--- I know this syntax is not supported
group_name="dbt_post_process",
compute_kind="python",
)
def something():
???
rex
05/10/2023, 4:36 PM[dbt_assets] = load_assets_from_dbt_project(...)
dbt_asset_keys = list(dbt_assets.keys_by_output_name.values())
@asset(
non_argument_deps=dbt_asset_keys,
...
)
def something():
...
Romain Robert
05/10/2023, 4:47 PM