https://dagster.io/ logo
#ask-community
Title
# ask-community
d

David Weber

08/17/2023, 11:49 AM
Hi there! I am currently integrating our dbt project in Dagster. Dagster assets handle all our data ingestion, and store the data on a database. And then the modelling on that database is done with dbt. That means: The output of the last dagster ingestion asset is marked as a"source" in dbt. Is there a way, that the lineage shows this last Dagster ingestion asset as source for dbt, instead of having a separate asset for the dbt source? To demonstrate the problem, here is the code: First, I load all dbt resources as assets:
Copy code
@dbt_assets(manifest=DBT_MANIFEST_PATH)
def dbt_assets(context, dbt):
    yield from dbt.cli(["build"], context=context).stream()
Then, to get the lineage right, I make the dbt source as a Dagster asset, which has the dependency on the last Dagster ingest asset.
Copy code
@asset(key=get_asset_key_for_source([dbt_assets]), "MY_DBT_SOURCE")
        , non_argument_deps={"LAST_DAGSTER_INGEST_ASSET"}) # dependency to the dagster asset, that outputs the table we want as source
def dbt_source_asset():
    pass
But the issue is that by doing this, the dbt source "MY_DBT_SOURCE" is modelled as a separate asset, and also respectively part of the dependency graph. I would like to avoid the dbt source being a separate dagster asset, as it is basically just the output of the last Dagster ingest asset... But in the above
@dbt_assets
I cannot put an argument
non_argument_deps
, which would make it possible that for these dbt models, a Dagster asset is their source. How can I solve this? How could I point the dbt source to the last dagster ingest asset? I havent found any solutions to this in the documentation or in the dagster-dbt Tutorial... Thanks!!
🤖 1
r

rex

08/17/2023, 1:03 PM
d

David Weber

08/17/2023, 1:43 PM
Thanks @rex!
But I had some weird behaviors with specifying dagster asset keys directly in dbt yml VS using a CustomDagsterDbtTranslator. I will check this again and probably reach out with another post on that.
r

rex

08/17/2023, 3:48 PM
Got it. If you run into any issues, please let us know. Happy to make amends to the documentation if anything is confusing.