https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
s

sandy

06/29/2023, 6:25 PM
Next in my series of polls about how you use dbt: how do you handle your profiles.yml? 1️⃣ - check it in to my git repository 2️⃣ - put it in
~/.dbt/
If your answer is 1️⃣ , what folder do you put it in? The root of your dbt project? Nested under a
config/
folder? Somewhere else? And does this cause any problems if developers want to customize their profiles for local develop? If your answer is 2️⃣ , how do you handle deployment? I.e. do you bundle it with your Docker image? And, if so, where do you pull it from to get it in there? Not going to bother everyone with an
@channel
this time, but would still love anyone and everyone's input.
2️⃣ 1
1️⃣ 16
l

Leo Qin

06/29/2023, 7:46 PM
we have a directory in our repo that contains two dbt projects in their own directories, the
profiles.yml
lives in that directory. Haven't had many complaints from developers re customization, but we heavily use env vars with reasonable defaults
👀 1
a

Adam Bloom

06/29/2023, 8:20 PM
A mix of these, actually. In our
dbt
directory, we have
profiles.yml
with a single target that is almost entirely parameterized with env vars, so dagster can use it directly. For local development, you can either set the env vars, or override the dbt profiles directory and specify a profile.
👀 1
g

Gabriel Montañola

06/29/2023, 10:30 PM
• nested inside a folder • plenty of targets + envvars
👀 1
v

Vinnie

06/30/2023, 7:53 AM
Porque no los dos? • within
dbt_project/profiles/profiles.yml
, this is how Dagster runs dbt, it’s bundled with the deployed docker image for our dbt code location. Sensitive info is stored in env vars, fetched from secrets in the local docker-compose setup that’s used if we want/need to test something end-to-end in a setup roughly mirroring our prod deployment • devs can still run everything without going the dagster route if they setup the profiles themselves which would create a
~/.dbt/profiles.yml
file
😅 1
👀 1
p

Patricia Goldberg

07/04/2023, 2:10 PM
Same as Vinnie for me
2 Views