https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jayme Edwards

08/20/2022, 12:19 AM
Hey y'all, I'm using the helm chart for dagster, and I've built a docker image for the user repository. It works fine when calling regular ops, but I also have DBT chained to it. The issue I'm running into, is how to get a profiles.yaml file with the right settings onto the docker image for my user repository. It has to be dynamic in that the connection string settings will differ in my staging and production environment, but I want to reuse the same docker image obviously. The helm chart for dagster has volume and volumeMounts as settings in values.yaml, but when I set these it doesn't seem to do anything. I was looking at the source for dagster's helm templates and I don't see these custom volume/mount values being referenced anywhere, so maybe they aren't functional currently? It looks like you also can't mount local hostpaths either, which is a bummer I'd like to be able to do that for running out of k8 locally while developing. Anyone figure out a way to get the profiles.yaml for dbt setup in their docker image running in k8 with the helm package for dagster? Or any tips? Thanks!
dagster bot responded by community 1
a

Adam Bloom

08/20/2022, 12:30 AM
we're doing this with env vars and k8s secrets.
as an example, we have entries like this in our profiles.yaml:
Copy code
host: "{{ env_var('DBT_DB_HOST') }}"
      port: "{{ env_var('DBT_DB_PORT', 1234) | as_number }}"
https://docs.getdbt.com/reference/dbt-jinja-functions/env_var Looking further in the docs, there are some special things enabled if you prefix the env vars with
DBT_ENV_SECRET_
, so maybe do that
the user code helm chart defines an
envSecrets
parameter you'll need to tie this all together
j

Jayme Edwards

08/20/2022, 2:31 AM
Thanks @Adam Bloom I didn't know you could replace values in files like that, I'm still kind of new to k8.
a

Adam Bloom

08/22/2022, 3:55 PM
That's a dbt specific implementation, but yeah - lots of opportunities to use env vars instead of hard coding things!
6 Views