Hi! How would I set up a Dagster resource so that ...
# ask-community
a
Hi! How would I set up a Dagster resource so that I can have a configured instance of it where configuration is taken from environment variables if they are provided otherwise it takes default or simply None value?
Copy code
from dagster import resource, Field, StringSource

@resource(config_schema={
    "user": Field(StringSource, description="Username."),
    "pass": Field(StringSource, description="Password."),
    "proxy": Field(StringSource, is_required=False, description="HTTP proxy to use if provided."),
})
def service_credentials(context):
    """S3FileSystem Dagster Resource."""
    return context.resource_config

service_credentials_from_env = service_credentials.configured({
    "user": {"env": "SERVICE_USER"},
    "pass": {"env": "SERVICE_PASS"},
    "proxy": {"env": "SERVICE_HTTP_PROXY"},
})
If I do the case above it simply errors with
Copy code
dagster._core.errors.DagsterInvalidConfigError: Error in config for resource service_credentials
    Error 1: Post processing at path root:config:proxy of original value {'env': 'SERVICE_HTTP_PROXY'} failed:
dagster._config.errors.PostProcessingError: You have attempted to fetch the environment variable "SERVICE_HTTP_PROXY" which is not set. In order for this execution to succeed it must be set in this environment.
🤖 1
Ah, it seems even default value fallback is not supported. This really takes away the value from StringSource, IntSource and BoolSource IMO. https://github.com/dagster-io/dagster/issues/3947
s
Hi Anton, I agree this is a problem-- I assigned this issue to myself and will try to knock it out in the next couple weeks.
❤️ 3
a
Thanks Sean!