What's the best way to add resources to the contex...
# announcements
What's the best way to add resources to the context at runtime? Trying to avoid passing a parameter to every solid in a pipeline. For example, I'd like to have a
path_to_object: my_object.yaml
element in the config of my first solid that adds "my_object" to resources and can be accessed via
in all later solids.
👀 1
Hi Ben - I don't believe this is currently possible. The purist perspective would be that this is an anti-pattern - the equivalent of mutating global state. Conceptually, there's a data dependency between the solid constructing "my_object" and the solids consuming "my_object", and ideally that dependency would be modeled through inputs/outputs between solids like other data dependencies. Apologies if that's not the most satisfying answer. If this comes up a lot, it could be worth APIs to make this pattern easier to implement.
Fair enough - I can see how the approach could be abused. Don't want a magic global pattern, want a way for the Dagit user to load parameters from a configuration file at the start of a pipeline. Build my own hydration function?
If all you need is a value that is accessible by all solids and set in config once then a resource is want you want
For example:
Copy code
def test_dict_resource():
    from dagster import pipeline, resource, Permissive, execute_pipeline, ModeDefinition

    def dict_resource(init_context):
        return init_context.resource_config

    def consumes_the_dict(context):
        assert context.resources.the_dict == {'a': 'b'}

    @pipeline(mode_defs=[ModeDefinition(resource_defs={'the_dict': dict_resource})])
    def pipe():

    execute_pipeline(pipe, run_config={'resources': {'the_dict': {'config': {'a': 'b'}}}})
so all that “resource” does is take a config value specified in the run config (in this case {‘a’: ‘b’}) and makes it available to all solids.
Perfect, that's exactly what I'm looking for. Thanks for the code starter, I'll wrap my head around it.
👍 1