I am trying to better manage secrets in K8s. Does anyone have a good solution for substituting env vars into IOmanager configs using the new config style? I prefer to load the config from a yaml using config_from_files() and then using the configs to initialize the IOManagers using Pydantics deprecated parse_obj(). However, this won't work for environment variables since I can't do the {'env':value} trick in the yaml file because config_from_files() does not substitute them. In the old dagster implementations {'env':xxx} would get substituted in the IOManager.configured() method. What is the best way to do this now from a yaml file without resorting to coding it in python using EnvVar()? I saw that Pydantic's solution is pyndantic_settings but that might not work here if I combine yaml file and env vars?

For secrets management, we are having success with vault. These can then be set independently to staging or prod values. We then load these into Dagster with

from dagster import EnvVar

Further to this, there is documentation here on k8s secret management.

Thanks @Chris Roth, I found a solution for now. I will also look at using a vault during our next phase. Thanks for pointing me in that direction.