Good day everyone! I would appreciate an input on a problem I currently have. I'm running dagster 14.14 on k8s and I need to access ops resource from io manager. This is what I try to do:

@io_manager(required_resource_keys={"k8s_pg"})
def my_iomanager(init_context):
    return ioClass(
        resource=init_context.resources.k8s_pg
    )

Where ioClass has init method

self.resource=resource

. The problem I'm having is that the resource inside ioClass is initialized without the env vars. However, the hard coded parameters of the resource are there. I observe this by logging from inside the ioClass

logging.critical(_self_.resource.__dict__)

, resource params with

os.getenv('xxx')

are None. I've also verified that all the required env vars exist inside my user depl container by executing the shell and running

printenv

. The code that I have works if I execute from localhost with

dagster job execute ....

. I've also tried adding _`executor_def=in_process_executor`_ to my job decorator which didn't change anything. Any help would be appreciated!

Hi @iskander - it sounds like you may need to include some secrets or configmaps on your run launcher in the helm chart in order to add the right environment variables: https://github.com/dagster-io/dagster/blob/70b6495449826ced71f1fac4fa8775e01fae0de8/helm/dagster/values.yaml#L418-L442 This has been improved in later versions of dagster, so upgrading is another option - secrets and configmaps on your user code deployment are now automatically passed through to the launched runs as well

Hi @daniel, thank you for your response! I fixed it! Here's how - I'm running on rancher and I have already included all the necessary envSecrets correlating to rancher Secret names. I then verified that the key:value pairs of a new rancher secret were present in the user deployment container. My mistake was in assuming that the same env vars from the user depl container will be passed to the k8s Job.... Which they weren't! For some reason, the latest rancher Secret that I've added wasn't present in the k8s Job. So the solution was to add the necessary key:value pairs representing the env vars to another rancher Secret. That solved it. Could this be a dagster issue? P.S upgrading to the latest release is on my radar as a todo in the near future.

That's a feature that's been added in later dagster versions, yeah (including env vars from the user deployment in the pod for the launched run). Glad its workign!