https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
p

Paul Burns

07/21/2023, 3:00 PM
Hello! Is it possible to configure Dagster where any attempt to materialize a dbt asset that is already in the process of being materialized is ignored/discarded?
👍 1
d

Duke

07/21/2023, 3:22 PM
same concern. right now I think the run just fails (because dbt creates some
*__dbt_backup
relation in the other run) and gets re-executed later. But I hope Dagster would be able to skip the materialization to save compute. This is especially important when multiple auto-materialize runs could trigger same subset of assets
p

Paul Burns

07/21/2023, 5:14 PM
Yeah we are just in the design faze but it would be super helpful for us. We at times have a model run twice at the same time and not only is that more cost but it can create incorrect data with a custom dbt materialization we have.
c

claire

07/21/2023, 8:49 PM
Hi Paul. This is not currently possible, but we have discussed similar functionalities (i.e. skipping materializations when the upstream input and the code has not changed). I'd recommend filing an issue
p

Paul Burns

07/24/2023, 4:01 PM
I will file an issue!
2 Views