The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

What's the best way to add resources to the context at runtime? Trying to avoid passing a parameter to every solid in a pipeline. For example, I'd like to have a `path_to_object: my_object.yaml` element in the config of my first solid that adds "my_object" to resources and can be accessed via `<http://context.resources.my|context.resources.my>_object` in all later solids.

Hi Ben - I don't believe this is currently possible.  The purist perspective would be that this is an anti-pattern - the equivalent of mutating global state.  Conceptually, there's a data dependency between the solid constructing "my_object" and the solids consuming "my_object", and ideally that dependency would be modeled through inputs/outputs between solids like other data dependencies.

Apologies if that's not the most satisfying answer.  If this comes up a lot, it could be worth APIs to make this pattern easier to implement.

Fair enough - I can see how the approach could be abused. Don't want a magic global pattern, want a way for the Dagit user to load parameters from a configuration file at the start of a pipeline. Build my own hydration function?

If all you need is a value that is accessible by all solids and set in config once then a resource is want you want

For example:

```def test_dict_resource():
    from dagster import pipeline, resource, Permissive, execute_pipeline, ModeDefinition

    @resource(config_schema=Permissive())
    def dict_resource(init_context):
        return init_context.resource_config

    @solid(required_resource_keys={'the_dict'})
    def consumes_the_dict(context):
        assert context.resources.the_dict == {'a': 'b'}

    @pipeline(mode_defs=[ModeDefinition(resource_defs={'the_dict': dict_resource})])
    def pipe():
        consumes_the_dict()

    execute_pipeline(pipe, run_config={'resources': {'the_dict': {'config': {'a': 'b'}}}})```

so all that “resource” does is take a config value specified in the run config (in this case {‘a’: ‘b’}) and makes it available to all solids.

Perfect, that's exactly what I'm looking for. Thanks for the code starter, I'll wrap my head around it.