The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

running into what appears to be a bug trying to test preconfiguration of a resource using `StringSource` fields that draw from an environment variable. reproduction code in the thread.

```@dataclass
class Thing:
    foo: str


@resource({
    'foo': Field(StringSource),
})
def thing(init_context):
    return Thing(init_context.resource_config['foo'])


@configured(thing)
def configured_thing(_config):
    return {
        'foo': {'env': 'FOO_VAR'}
    }


resource_context = InitResourceContext(resource_config={}, resource_def=configured_thing)
configured_thing.resource_fn(resource_context)
"""
Traceback (most recent call last):
  File "test.py", line 29, in &lt;module&gt;
    configured_thing.resource_fn(resource_context)
  File "test.py", line 18, in thing
    return Thing(init_context.resource_config['foo'])
KeyError: 'foo'
"""```

behavior is identical regardless of whether the environment variable is defined (or empty vs nonempty)

is this because `StringSource` only works for solids/resources that take a single parameter?

hey Steve, so the reason this doesnt work is that InitResourceContext expects that all the config for a resource has already been resolved when you pass it in

as in, it expects that our config system has already resolved "foo" to be some value by the time `InitResourceContext` is initialized. In general, we consider `InitResourceContext` to be an internal API, but the `build_resources` API that Alex mentioned earlier will handle this case

got it. so when dagster spins up the resource context during a pipeline run, it'll properly resolve the env vars?

great. i'll see if i can adapt my unit tests to use the `build_resources` api

Sure, let me know if you have any questions or run into any problems!

I do just want to clarify that the API is still being actively developed, so you may run into some breaking changes in the near future. If that is too risky, then you could always spin up a fake pipeline for the express purpose of instantiating resources.

<@U01PHS45686> you can just call `os.environ.get('FOO_VAR')` directly in the `configured` function

yeah, that's what we had been doing in the past, i just figured it was a better practice to use dagster's built-in mechanisms for that

i'm fine with keeping up with an experimental api for unit tests, since this won't be run in production

ah, when alex mentioned this was going to be "released" in the coming months, i assumed they meant 'released in its final form,' i didn't realize that `build_resources` itself wasn't available in 0.10.9

looks like spinning up a pipeline is the way to go for now - moving us up to bleeding-edge dagster might indeed be too risky for the team