https://dagster.io/ logo
x

Xu Zhang

11/13/2020, 8:15 PM
another side question, i’ve managed to attached dagit onto a customized version of Flask that can be deployed into our company’s host, with all default configs; is there a way to config the dagit in code directly instead of reading from a YAML? reason is that the DB credentials are encrypted in another system which can only be accessed when the app is running, we can’t store plain credential settings inside a YAML.
@daniel i guess this can be solved by the information you provided in another thread:
Copy code
Or if you want to be able to use a run launcher to execute your pipeline in a different process like dagit does, we have an option for that as well, it's just a little bit more hidden currently
that
hidden option
d

daniel

11/13/2020, 8:22 PM
I think that's something different - it sounds like you want a different way to set up your DagsterInstance than loading it from a YAML file
s

sashank

11/13/2020, 8:27 PM
can you store the credentials in environment variables?
if so, you can configure the yaml to read those environment variables
x

Xu Zhang

11/13/2020, 8:29 PM
@sashank that is a good idea, i think i should be able to do that
s

sashank

11/13/2020, 8:30 PM
if so, you can usually use
env: ENV_VAR_NAME
in places that expect a string
for example:
instead of:
Copy code
event_log_storage:
  module: dagster_postgres.event_log
  class: PostgresEventLogStorage
  config:
    postgres_db:
      username: my_username
      password: my_password   
      hostname: my_host      
      db_name: my_db
      port: 5432
you can do:
Copy code
event_log_storage:
  module: dagster_postgres.event_log
  class: PostgresEventLogStorage
  config:
    postgres_db:
      username:
        env: DAGSTER_PG_USERNAME
      password:
        env: DAGSTER_PG_PASSWORD
      hostname:
        env: DAGSTER_PG_HOST
      db_name:
        env: DAGSTER_PG_DB
      port: 5432
and we’ll automatically load those values from the environment variable instead
x

Xu Zhang

11/13/2020, 9:21 PM
Got it, thank you!