https://dagster.io/ logo
Join Slack
Powered by
Hello all. I am trying to add my postgres storage ...
# ask-community
k
Hello all. I am trying to add my postgres storage credentials to environment variables as mentioned in docs: https://docs.dagster.io/deployment/dagster-instance#dagster-storage And both the 
dagster.yaml
and 
.env
are set in DAGSTER_HOME dagster.yaml 
storage:
postgres:
postgres_db:
username:
env: DAGSTER_PG_USERNAME
password:
env: DAGSTER_PG_PASSWORD
hostname:
env: DAGSTER_PG_HOST
db_name:
env: DAGSTER_PG_DBNAME
port: 5432
But when I try to run, I receive the below error: 
Error 3: Post processing at path root:postgres_db:hostname of original value {'env': 'DAGSTER_PG_HOST'} failed:
dagster._config.errors.PostProcessingError: You have attempted to fetch the environment variable "DAGSTER_PG_HOST" which is not set. In order for this execution to succeed it must be set in this environment.
d
Hi - .env files only work for environment variables referenced in your code. It does not affect environment variables in your dagster.yaml (this is a common point of confusion though)
k
Hello Daniel, Thank you for the answer but in the documentation (in the one I shared in my original message) it was mentioned that you can retrieve the values from the environment variables using 
env:
key.
d
We’ll get those docs fixed - sorry for the confusion. The .env file works for secrets that are reference in your code, but not in your dagster.yaml file
You will need to set those environment variables that you reference in your dagster.yaml file using some other method
You can still use the env key in your dagster.yaml file though - it’s just that you can’t put the values in the .env file
k
Thank you
Ok.... I found the answer. Apparently, it is the same way that you set the DAGSTER_HOME to be able to run dagster-daemon run (which i think is missing from the docs... It just say what to do in the error message), unless I missed it from the docs So, to be able to use the env variables in a windows machine I need to run the below commands in a Powershell: 
$env:DAGSTER_PG_HOST= 'my_host'
$env:DAGSTER_PG_DB= 'my_db'
$env:DAGSTER_PG_USERNAME= 'my_username'
$env:DAGSTER_PG_PASSWORD= 'my_password'
And use the below in dagster.yaml (as mentioned in the docs) 
storage:
postgres:
postgres_db:
username:
env: DAGSTER_PG_USERNAME
password:
env: DAGSTER_PG_PASSWORD
hostname:
env: DAGSTER_PG_HOST
db_name:
env: DAGSTER_PG_DB
port: 5432
It would be good to mention it in the documentation 😁
6 Views
Open in Slack
PreviousNext
Powered by