Hi folks, I’d like to share some configuration across a handful of solids. What’s the best way to do this? Conceptually, creating a new resource would fit; however, it seems a bit like overkill to me, since the desired configuration is very simple (a list of integer IDs).

i've just put the shared config schema dictionary in a top-level constant and referenced it in the definition of each solid

Thanks, Steve - Do you still have to specify the config values in multiple places in the YAML with this approach?

you can extend it with additional values for each one with the splat operator, e.g.

SHARED_SCHEMA = {
  'foo': StringSource,
}

@solid(config_schema={
  **SHARED_SCHEMA,
  'bar': IntSource,
})
def foobar(self): ...

Thanks, Steve. Avoiding duplicating the values in the run config is what I’d like to avoid if possible. Since it could lead to some unexpected edge-cases and errors. Perhaps creating a resource or config-loader solid is the simplest approach here 😕

Hey @Ben Torvaney - I believe that creating a resource is best approach for what you're trying to do. Do you have an idea in mind of what it would look like to make this simpler?

Thanks, Sandy. I don’t really have a clear idea, no. One option could be to enable application of config to multiple solids:

solids:
  # ...:
  custom:
    solids:
      - my_solid_a
      - my_solid_n
    config:
      some_config_ids:
        - 12
        - 90
        - 42

But I have to say, I’m not convinced this is a great idea… My hesitance towards resources is that it has additional flexibility that I don’t think would ever be used. Perhaps that’s more of an aesthetic concern than a practical or conceptual one?

I had to do something similar, the most straightforward solution i could find was a resource that returned a class/dataclass containing the “global” config values

there is also

configured

, if these values rarely change

yeah, for static values loaded from environment variables i've been using

configured