Hello again, team! How do you usually provide sens...
# ask-community
Hello again, team! How do you usually provide sensitive resources configuration in case of schedules/sensors? Example: We have dev and prod clusters running Dagster, and we want to use different base urls for a http client resource. That's easily resolved in Dagit as we have separate instances of it for dev and prod. But in case of schedules/sensors the configuration is hardcoded. I could use
in schedule definition, but I can't do that within Dagit, which will also lead to some approach discrepancy.
dagster bot resolve to discussion 1
Hi Nick, you can use a StringSource type that tells Dagster to load config using an env var, e.g. here's an example:
Copy code
        "foo": Field(StringSource)
def my_op(context):
    return context.solid_config["foo"]
Then you could launch the run with e.g this config.
Copy code
Then you need to make sure that MY_ENV_VAR_NAME is present in the environment where the run is launched (e.g. by configuring your run launcher to include it)
@daniel thanks a lot for your answer, I've just tested that. Some notes: • Not all the values for Dagster provided resources are typed as StringSource, e.g.
which is Field(str). I think that can overcome that with
on my side though... • The original problem is a kind of discrepancy between scheduler configuration and dagit one. I can tweak dagit configuration on the fly, e.g. i can add or remove values between runs. Good example is:
Copy code
Usually we need this parameter only for local development with minio. We can easily add it for local dagit and avoid it for production dagit (where we don't even have this env variable set). But that becomes a real problem when we are trying to configure
with the same piece of configuration, as its will be bundled into the code, and the env variable will become a requirement.
Got it - it's true that there isn't really a way to change schedule configuration on the fly currently. I could imagine building something on top of the current system that pulls the config from a local YAML file or something rather than from code, and swapping in a new version of that file or something - we don't have anything like that built-in currently though