https://dagster.io/ logo
#announcements
Title
# announcements
c

Cris

05/08/2020, 1:19 AM
Hi all, I would like your opinion in the best way to set up settings that some solids in my pipeline use. 1. Use env variables directly,, it may reduce code maintainability and I wouldn't be sure whether this option could work in distributed settings. 2. Set up a settings.py with centralized variables that can be read from env or a yaml. However, I dont know how to make dagster load this module before the solids run. 3. Create a settings resource, Is there a way to make it available to all solids and resources? 4. Maybe another option?
a

alex

05/08/2020, 1:45 AM
I would look in to using solid config, with the
StringSource
type in particular. This type works by either taking the value directly or as
env: ENV_VAR_TO_LOOKUP
to look up the value from the environment
a resource is a good idea if you need to share the config values repeatedly - you just need to make sure all the solid declare the resource as required via
required_resource_keys
.
c

Cris

05/08/2020, 2:30 PM
the stringsource seems nice. Could you point me to the documentation?
3 Views