https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
l

Leo Qin

05/09/2023, 11:49 PM
hi all, i'm noticing that when a test fails, the asset associated with the test still records a materialization event - is this expected behavior?
r

rex

05/10/2023, 9:49 PM
Yeah. I am assuming you are running
dbt build
, which essentially runs
dbt run
and
dbt test
together (link). we currently output
AssetMaterialization
and
AssetObservation
events using the run results and provided manifest. What behavior were you expecting here?
f

Félix Tremblay

05/12/2023, 1:22 PM
@Leo Qin It is dbt's normal behavior. If you are looking for blue/green deploy, it can be done but it needs to be configured. There's a post about it in dbt discourse
@rex do you think dagster-dbt could provide better support to help users implement blue/green deploy? A Tutorial in the docs would be a great start. Ultimately, Dagster could handle much of the logic by itself
r

rex

05/12/2023, 1:27 PM
I assume that Leo is talking about the materialization event behavior in Dagster. As Félix mentioned, this is standard dbt behavior, that the model is materialized first in the data warehouse, and then tests are run. Even if the tests fail, the model is still materialized. In Dagster, we create an
AssetMaterialization
that corresponds to a dbt model materialization. Currently, the behavior is that this
AssetMaterialization
is at the same time a dbt model is materialized, which is essentially mimicking the behavior you see in dbt.
Yeah we’re looking to support these custom patterns more thoroughly soon. The basis of this support is that we’ll let you generate software defined assets for your dbt models out-of-the-box, but you can also specify the compute function that generates these assets. Think of it as a custom
@multi_asset
decorator, but for the
dagster-dbt
. Once we expose the compute function to y’all, you can do what you want to achieve your use cases e.g. • blue/green deploy, • run arbitrary dbt commands together, like
dbt run
+
dbt test
, instead of just
dbt run
/
dbt build
• use dbt together with Slack/S3/arbitrary resources • etc
f

Félix Tremblay

05/12/2023, 1:59 PM
Hey @rex that sounds like a very powerful feature for advanced use cases. I'm wondering if that might be a bit intimidating for novice users. Do you think that Dagster could include a tutorial for blue/green deploys with dbt?
r

rex

05/12/2023, 2:05 PM
Yeah I think we want to strike a balance here: 1. We want to empower our users to be able to take full advantage of the frameworks at play here (dagster + dbt) 2. But yes, for canonical and established use cases for dbt (like the blue/green deploy you mentioned), I think we will have tutorials to show how this works in practice with (1).
👌 1
l

Leo Qin

05/12/2023, 2:27 PM
Oh interesting - yeah, I was investigating it because I was writing some notifications enhancements, but now that you mention it, the adapter we use allows for zero-downtime builds, which i think can be posed as some variation of a blue/green deploy