Stefan Adelbert12/09/2021, 3:10 AM
. However, when I click
in dagit for a job's run I don't see the configured config values used for the resource for that run. I also don't have the ability to override the config values for a configured resource when launching a new run in dagit. I suspect it's because
creates a new resource from an existing resource, "hardcoding" the config values; therefore the new resource has no config. Could someone suggest a way to preconfigure a resource (or op) such that: • the config is evident in dagit for an historic run • the config can be inspected and overridden for a new run
chris12/09/2021, 8:09 PM
API doesn't provide a default configuration, it actually completely overrides the config schema with the provided hardcoded value / config mapping. I think the only way to achieve what you're looking for is to provide default values to your config schema directly (see the Field API)
Stefan Adelbert12/09/2021, 9:34 PM
might need one default config value in one scenario and a different default config value in another scenario. The config schema is intrinsically linked to the
and cannot be overridden, which is somewhat similar to my earlier Resource to Resource Dependencies post.
chris12/09/2021, 9:39 PM
Kind of jank, but it would get the job done. I'm not 100% sure how the new config schema would populate, but I'm pretty sure the outer config schema would populate the default fields in the launchpad for example.
@op(config_schema=some_original_schema) def my_op(): ... def config_fn(outer_config): return outer_config def redefine_config_schema_with_defaults(schema): ... my_op.configured(config_or_config_fn=config_fn, config_schema=redefine_config_schema_with_defaults(some_original_schema))