Stefan Adelbert
12/21/2021, 4:44 AMdagster.yaml
I have previously had something like the following in my dagster.yaml
which allowed me to use environment variables to control some details of how a run worker will log,
python_logs:
dagster_handler_config:
handlers:
gcpHandler:
(): my_logging.Handler
level: DEBUG
formatter: gcpFormatter
formatters:
gcpFormatter:
(): my_logging.Formatter
client:
env: DAGSTER_CLIENT
environment:
env: DAGSTER_ENVIRONMENT
It worked really nicely.
I switched back to using this technique today, but the variable substitution doesn't seem to be working (0.13.12) - the client
and environment
parameters to my_logging.Formatter.__init__
are dicts like {"env": "DAGSTER_CLIENT"}
, as if the variable substitution has been skipped.
I am using variable substitution in other parts of the dagster.yaml
and that seems to be working fine, e.g.
compute_logs:
module: dagster.core.storage.local_compute_log_manager
class: LocalComputeLogManager
config:
base_dir:
env: DAGSTER_LOCAL_COMPUTE_LOG_MANAGER_DIRECTORY
I'm hoping that I'm simply doing something stupid, but I can't figure it out...