David Weber
08/17/2023, 11:49 AM@dbt_assets(manifest=DBT_MANIFEST_PATH)
def dbt_assets(context, dbt):
yield from dbt.cli(["build"], context=context).stream()
Then, to get the lineage right, I make the dbt source as a Dagster asset, which has the dependency on the last Dagster ingest asset.
@asset(key=get_asset_key_for_source([dbt_assets]), "MY_DBT_SOURCE")
, non_argument_deps={"LAST_DAGSTER_INGEST_ASSET"}) # dependency to the dagster asset, that outputs the table we want as source
def dbt_source_asset():
pass
But the issue is that by doing this, the dbt source "MY_DBT_SOURCE" is modelled as a separate asset, and also respectively part of the dependency graph. I would like to avoid the dbt source being a separate dagster asset, as it is basically just the output of the last Dagster ingest asset...
But in the above @dbt_assets
I cannot put an argument non_argument_deps
, which would make it possible that for these dbt models, a Dagster asset is their source.
How can I solve this? How could I point the dbt source to the last dagster ingest asset?
I havent found any solutions to this in the documentation or in the dagster-dbt Tutorial...
Thanks!!David Weber
08/17/2023, 12:09 PMrex
08/17/2023, 1:03 PMDavid Weber
08/17/2023, 1:43 PMDavid Weber
08/17/2023, 1:44 PMrex
08/17/2023, 3:48 PM