Is load assets from dbt project or dbt cli resource recommen

Question

Is load assets from dbt project or dbt cli resource recommended I am currently using load assets from dbt project and observing DagsterInvalidDefinitionError Group name already exists on assets for the DBT specific assets and feel lost debugging this as I did not manually add the DBT assets to any group except one which is inferred from the package location

geoHeil · Answer

package group name dbt my first dbt model package group name dbt my second dbt model

geoHeil · Answer

to me this looks like a dagster bug where load assets from dbt project and I are setting the group

geoHeil · Answer

```models target Config indicated by + and applies to all files under models example example +materialized view```

geoHeil · Answer

Turns out dagit is tryingto hape the `example` directory to an asset group which is colliding with my manual mapping 1 when the names are different 2 when they are the same as it is only allowed to set the group once

geoHeil · Answer

assuming I have a python package `foo` and `bar` which should both show up in dagit as asset groups named foo and bar and a `sql dbt` directory with a sub folder also for models specific to foo and bar how can I get the respective python and DBT assets to show up in the same asset group

geoHeil · Answer

all foo in foo all bar in bar

geoHeil · Answer

Furthermore the source DB of the asset apparently also must match the asset group otherwise dagster is generating no or only missing lineage

sandy · Answer

< owen> mind taking a look

owen · Answer

hi < geoHeil> right now the load assets from dbt project assigns a group name based on the directory that each model resides in and this means that applying a group afterwards will result the error you re seeing this behavior can and should be made configurable which we can likely get in for this week s release for the bit about source assets and incorrect lineage that definitely seems like a bug are you applying an asset key prefix to the dbt assets while after you load them

geoHeil · Answer

What I observe is that when the `source yaml` specifies `mydatabase mytable` the `mydatabase` is translated to an implicit asset group which may or may not exist even when not specifying any additional keys when loading the DBT assets

owen · Answer

hm I don t think I m fully following the expected behavior in my mind would be for an AssetKey to be generated to represent that source and it would be `AssetKey mydatabase mytable ` in dagit this would show up as a source asset without a definition basically just some grey text rather than a box and this key would not be associated with any asset group is that matching up to what you re seeing or is something else happening

geoHeil · Answer

I think this is what I am seeing but my problem is not necessarily what I want i e I might want to keep asset group and asset key prefix separate i e it might not be called `marketing` both times In such a case then even though the data is flowing in a DAG from a z the lineage will be broken and a source asset will be created Assuming my assetgroup foo with prefix bar is writing data to a database in schema bar then the source yaml is bar mytable but dagit right now would also expect this to be in asset group bar