https://dagster.io/ logo
Title
c

chrispc

08/09/2021, 4:38 PM
Hi team! I was wondering if you could help me with this logic. I need to reuse the same logic I had in this solid but using different resource (I have several different postgres databases so the resource is the same but the name and credentials different). I am not sure how to parametrize this because the resource is called in the code with the name of the resource). I mean I would like to avoid have the same logic twice just because the resource changes.
c

chris

08/09/2021, 5:13 PM
So if I understand correctly, you want to be able to reuse the same solid multiple times in the same execution, but with the resource key mapped to different resource instances for each run?
c

chrispc

08/09/2021, 5:15 PM
Thank you @chris. I want it for each run and for each solid, Could you look the image under this?
c

chris

08/09/2021, 5:29 PM
Right, I see. So this is something we've talked about supporting in the past, where essentially, you can remap a resource key to a different one, without having to rewrite your solid. This issue tracks that https://github.com/dagster-io/dagster/issues/2112. Since that isn't currently supported, there are a few workarounds you could try. The first is to use the factory pattern to reconstruct the solid requiring each resource. Something like this:
def get_solid_for_resource_key(resource_key: str) -> SolidDefinition:
    @solid(required_resource_keys={resource_key})
    def _inner(context):
        return context.resources.__getattribute__(resource_key)

    return _inner
Another solution would be to have a solid that takes all the databases under different keys, and then toggle which one you want to use based on config.
@solid(required_resource_keys={"a", "b"}, config_schema={"which_one": str})
def my_solid(context):
    if context.solid_config["which_one"] == "a":
         my_db = context.resources.a
    else:
         my_db = context.resources.b