Hi all, I'm trying to configure my Dagster deployment in GKE to use an external postgres database hosted in Cloud SQL. I'm using Workload Identity and I'm wondering what's the best way to point the dagster values.yml to that postgres instance. Is it to configure a private IP address for the external instance and set the field below to that IP?
# set this PostgreSQL hostname when using an external PostgreSQL database
If so, is there a way to use a secret for this? Otherwise, is there a better way to connect?
09/09/2021, 2:38 AM
Would love to hear Dagster's team opinion on this.
In my case, I did need to put private IP of the Cloud-SQL there, and would use a script to plug that host along with username/password (from git-ignored file) into the helm-deploy command.
Eventually, I found that managing Cloud-SQL was too complex for our use case, so we started streaming logs into BigQuery ( using custom logger resource ) and use the helm-postgres instance instead.