https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
i

Ismael Rodrigues

02/08/2024, 9:36 PM
Night people. How you guys make dependencies between assets and dbt_models that you guys created? In my case I load the data into redshift using a asset and I want the others to trigger after that, but is being a challenging one because I can't find a way of doing it. Would someone mind to help me?
🤖 1
a

Adam Bloom

02/09/2024, 2:05 AM
Dbt is upstream or downstream? If it’s downstream, you need to use dbt sources (and potentially a custom asset key function in a dagster dbt translator class) to connect your dbt models to upstream assets. If dbt is upstream, you can just add the dbt asset keys as non argument deps of any downstream assets
i

Ismael Rodrigues

02/09/2024, 3:08 AM
Is downstream. I have this translator and the asset_key, but how can I achieve this connection? I've tried a lot of options, none of them worked
a

Adam Bloom

02/09/2024, 3:09 AM
so you have two assets in your asset graph, one that's the actual upstream asset, and one that is the source asset that should be the same one?
i

Ismael Rodrigues

02/09/2024, 3:10 AM
I have one asset that loads 4 tables and I want my dbt_model to depend on 2 of the tables it loads. The asset is multidimensional partitioned by entity | date
a

Adam Bloom

02/09/2024, 3:11 AM
any reason that you can't have each table in its own asset? that would probably help simplify this
i

Ismael Rodrigues

02/09/2024, 3:15 AM
I'm used to not creating one job/asset for every single thing I would need to load Usually I get data from databases, apis or S3 and as the code to get the data is the same my assets and jobs were made in a way that would run all the needed entities without having one for each. I'm acctually migrating to assets now, I was using only OPs and Jobs until now. I was hoping to keep my "pattern" of getting data using @graph_assets, only to avoid exactly the scenario of separating things And of course, I thought that would be easy to just adapt the downstream to keep things this way. That's the reason behind my headache today :')
But, as you said, I'm starting to understand that while using assets, I'll have to change my way of thinking
a

Adam Bloom

02/09/2024, 3:20 AM
you can definitely have a single op materialize multiple assets, if that is what makes the most sense.
i

Ismael Rodrigues

02/09/2024, 3:22 AM
yeah, I was thinking just that as a way of chaning things, keep everything as it is and just changing calls inside different assets, but it'll be a pain in the ass for one job that loads about 68 entities lmao
so, if a separate in two assets, my workflow will work just fine, right?
I'll name them inside sources.yml and the rest works like magic
a

Adam Bloom

02/09/2024, 3:24 AM
you might need to use the translator if the names in the sources.yml don't exactly match your asset keys, but that should link them up in your asset graph, yes
dagster yay 1
i

Ismael Rodrigues

02/09/2024, 3:25 AM
I'm already using one, thanks for the time Mr. Adam
2 Views