Hello Dagster community, I have multiple dbt proje...
# ask-community
e
Hello Dagster community, I have multiple dbt project and they all depend on each other. Unfortunately on the lineage graph all model sourced in another dbt project in duplicated without any description.. Is there a way to link all the dbt model to have on enormous but complete lineage graph ?
t
Hi! Let's assume you have Project A and Project B. Project B's sources are models produced in Project A. In your case, the reason why models are duplicated in Dagster is because the asset keys for Project B's sources aren't the same as the asset keys for Project A's models. If the asset keys are the same, then Dagster will figure out that those two are related and merge them. They might not be the same because each asset can have prefixes on top of their normal key, like `project_a/orders`and
project_b/orders
The solution to your problem would be overriding Dagster's default way of making those asset keys for each of your projects and controlling to make sure that each source's asset key matches the model's asset key. Depending on your setup, you might want to: • load each project separately with
dbt_assets
• define a separate
DagsterDbtTranslator
for each project ◦ define each
get_asset_key
method to resolve Project B's source to match with the other asset key for Project A Here's some documentation on how you define your own DagsterDbtTranslator.