Hello, I need help to setup dagster on Kubernetes ...
# ask-community
t
Hello, I need help to setup dagster on Kubernetes with external postgresql. I am a bit lost. dagster will be a subchart of my chart. one subchart is to setup postgresql differently in this K8s instance. Unfortunately Scretes structure are different and I am lost of how to setup cleanly the values.yaml To setup the external database I have this to setup the subchart named postgresql:
Copy code
postgresql:
  parameters: 
    DATABASE_SERVICE_NAME: postgresql-service
    POSTGRESQL_DATABASE: somepgdatabase
    POSTGRESQL_PASSWORD: test
    POSTGRESQL_USER: test
this creates secrets with those keys:
Copy code
database-name
database-password
database-user
For the subchart of dagster I have this:
Copy code
dagster:
  postgresql:
    # set postgresql.enabled to be false to disable deploy of a PostgreSQL database and use an
    # existing external PostgreSQL database
    enabled: false

    # Used by init container to check that db is running. (Even if enabled:false)
    image:
      registry: "registry.default.svc:5000"
      repository: "repo/postgresql"
      tag: "latest"
      pullPolicy: IfNotPresent
    # set this PostgreSQL hostname when using an external PostgreSQL database
    postgresqlHost: "postgresql"
    postgresqlUsername: test

    # Note when changing this password (e.g. in test) that credentials will
    # persist as long as the PVCs do -- see:
    # <https://github.com/helm/charts/issues/12836#issuecomment-524552358>
    postgresqlPassword: test
    postgresqlDatabase: somepgdatabase
    # set postgresql parameter keywords for the connection string.
    # see: <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS>
    postgresqlParams: {}
    service:
      port: 5432
and finally:
Copy code
global:
  postgresqlSecretName: "postrgresql-secret-name"
which seems to define configmap which points to the dagster secrets https://github.com/dagster-io/dagster/blob/e26db9e21f64e265c74f60b61a67ea154e125959/helm/dagster/templates/helpers/_helpers.tpl#L184 What I want if possible is that from a single secrets it is mapped to both env in subcharts. is it possible ?
In fact, I wonder if the global value is needed for my case since I manage it outside dagster.
Copy code
# Whether to generate a secret resource for the PostgreSQL password. Useful if
# global.postgresqlSecretName is managed outside of this helm chart.
generatePostgresqlPasswordSecret: true
r
The docs on this are here: https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#configuring-an-external-database I think in your case, you would want
Copy code
global:
  postgresqlSecretName: "database-password"

generatePostgresqlPasswordSecret: false
t
Unfortunately no 😞 the key is imposed to
postgresql-password
and is not a variable. See line 58: https://artifacthub.io/packages/helm/dagster/dagster?modal=template&amp;template=deployment-daemon.yaml So I can change the secret name but I need to add another key (duplicated) in this secret to access the value. Like done with: https://artifacthub.io/packages/helm/dagster/dagster?modal=template&amp;template=secret-postgres.yaml Maybe I can ask for an enhancement ?
I think dagster impose this because of the chart of postgresql.
r
ah - when you mentioned
database-password
, I thought it was the name of your secret. Yes you’re correct - it’s currently hard coded to read from the
postgresql-password
key of whatever secret you pass to the helm chart
feel free to file a feature request for this
t
Ok. I see 21 occurences with defaults. I will look