https://dagster.io/ logo
Title
f

Félix Tremblay

10/20/2022, 8:15 PM
Hello 👋 We use both dbt CLI (for development) and dbt Cloud (mainly for production deployment, although we are open to using Dagster instead for production deployment). We want to use Dagster to orchestrate dbt, but don't know if we should go with the dbt integration or the dbt Cloud integration. Looks like we could use either one, right? Could you please help me understand the pros and cons?
❤️ 1
o

owen

10/20/2022, 8:55 PM
Hi @Félix Tremblay ! currently , if you’re interested in using software-defined assets (which we generally recommend), only the dbt-core integration supports these APIs. However, within a few weeks there will be feature parity between these two integrations, so at that point I would say that there will be no particular difference between those two options (so if you have dbt Cloud set up already it’d probably be easiest to go that route)
f

Félix Tremblay

10/21/2022, 1:15 PM
Hi @owen, thank you for your help. Yes, SDAs are definitely something we are interested in! :daggy-love: Another aspect I want to consider is the Branch Deployment workflow. Our _dbt_project_ is in the same GitHub repository as our (currently empty) _dagster_project_. I am looking for a seamless branch deployment experience (e.g. you are adding a feature that touches both the dagster and the dbt assets: during the development, both dagster and dbt are operating in concert within the dev environment/target, then during deployment, both operate in concert within the prod environment/target). Do you think the minimal-effort development/deployment workflow I'm looking for is achievable? If you have any recommendations to help us get there, it would be greatly appreciated.
o

owen

10/21/2022, 6:07 PM
This guide: https://docs.dagster.io/guides/dagster/branch_deployments has a lot of relevant info here, but the short answer is that this is not too tricky to set up. Quick answer is that when you're configuring your dbt_cli_resource, you would do something like
dbt_target = "dev" if os.getenv("DAGSTER_CLOUD_IS_BRANCH_DEPLOYMENT") else "prod"

dbt_resource = dbt_cli_resource.configured({..., "target": dbt_target})
👀 1
the guide has more info on how to set up the dagster-specific assets to touch a different schema
f

Félix Tremblay

10/21/2022, 7:13 PM
Thank you Owen
As I understand, we would need to use the dbt (cli) integration in order to set up the workflow
o

owen

10/21/2022, 7:17 PM
yep — we haven’t yet figured out all the details of how dbt Cloud assets + branch deployments will work but that is something we do plan on implementing
f

Félix Tremblay

10/21/2022, 7:17 PM
I'm curious if it will become possible to achieve the same with the dbt Cloud integration since you mentioned there would be feature parity in a few weeks
o

owen

10/21/2022, 7:20 PM
that’s definitely the goal but i mostly meant that there would be a SDA integration for dbt-cloud, wasn’t factoring in the branch deployment part into this (sorry for the confusion!)