https://dagster.io/ logo
#announcements
Title
# announcements
a

Alexis M

11/27/2019, 2:08 PM
Hello everyone, I am a user of dagster and I couldn't find a way to inject environment variables in my resources. Is there any documentation about this ? Thank you 🙂
a

alex

11/27/2019, 4:04 PM
Could you provide some more context on what exactly you want to do?
os.getenv
can be used in your @resource creation function
a

Alexis M

11/27/2019, 4:08 PM
Sure, I want to create a db resource which gets its credentials (user, db_name, etc...) from a .yaml file populated by env variables like postgres_db_name=${POSTGRES_DB_NAME} but it's not replacing the value with the environment value. Not sure if it is clear enough, if so I will give more details.
Maybe it is something to deal with in the preset creation ?
a

abhi

11/27/2019, 6:19 PM
when you say .yaml file, are you talking about a dagster specific yaml file (like a preset file that you are loading?), or is this some generic configuration yaml file with credentials that you would like to read with environment variables already pre-populated?
m

max

11/27/2019, 6:24 PM
Hi @Alexis M consider the following:
Copy code
import os

from dagster import Field, ModeDefinition, String, execute_pipeline, pipeline, resource, solid


@resource(config_field=Field(String))
def env_resource(init_context):
    return os.getenv(init_context.resource_config)


@solid(required_resource_keys={'dagster_home'})
def log_dagster_home(context):
    <http://context.log.info|context.log.info>(context.resources.dagster_home)


@pipeline(
    mode_defs=[ModeDefinition(resource_defs={'dagster_home': env_resource})]
)
def dagster_home_pipeline():
    log_dagster_home()


if __name__ == '__main__':
    execute_pipeline(
        dagster_home_pipeline, {'resources': {'dagster_home': {'config': 'DAGSTER_HOME'}}}
    )
a

Alexis M

11/27/2019, 6:24 PM
Yes @abhi its a dagster specific file
a

abhi

11/27/2019, 6:31 PM
Ah yeah, dagster doesn't pre-populate your preset yaml files at the moment. You need to do something like what @max mentioned above (IE specifying your environment variable names in your config and using the resource abstraction and
os.getenv
to access your environment variables)
a

Alexis M

11/27/2019, 6:34 PM
I understand, thank you guys for your help ! 👌
dagstir 1
4 Views