Using Dagster Cloud with k8s executor, where do I put things that would normally go in

$DAGSTER_HOME/dagster.yaml

? Looking for

managed_python_loggers

and

dagster_handler_config

in particular. I packaged my

dagster.yaml

into my Docker image and set

DAGSTER_HOME

but the custom log handling is not being applied.

You can customize the log handling in the Helm agent settings: https://artifacthub.io/packages/helm/dagster-cloud/dagster-cloud-agent?modal=values&path=pythonLogs

Thanks, I'm trying that out. If I'm setting the log configuration for the agent, then that will apply to all jobs, right? With Dagster Cloud we currently have one "prod" deployment, and only some jobs' container images will have the custom log handlers that I want to use. So maybe I should create another deployment + agent just for the jobs that have the custom handler? I tried doing this:

helm show values dagster-cloud/dagster-cloud-agent > values.yaml

Then I edited

values.yaml

and added in my logging:

python_logs:
  pythonLogLevel: INFO
  managedPythonLoggers:
    # Capture logs from all Python loggers into Dagster's UI.
    - root
  # Specify additional handlers for the console logs (which include all Python logs too).
  dagsterHandlerConfig:
    handlers:
      combinedJson:
        class: processing.common.log.JsonFileHandler
        level: INFO
      otelSpan:
        class: processing.common.log.OtelSpanEventHandler
        level: INFO

And finally tried to update the agent with those values:

helm upgrade    --install user-cloud dagster-cloud/dagster-cloud-agent    --namespace dagster-cloud  -f values.yaml

However, when I run (a) a job that doesn't have my custom handlers it still works fine, and (b) the job that does have those handlers in its image, they don't seem to take effect.

Hey Mark - just flagging that in both dagster open source and cloud this particular line could lead to slowdown if there are a lot of logs, we've seen a couple of users run into trouble after setting it:

managedPythonLoggers:
    # Capture logs from all Python loggers into Dagster's UI.
    - root

Yep, it does slow down the Dagit UI, but it's more useful to capture the logs than not. Also I'm trying to get all the logs from custom loggers as well as context.log into my custom handlers (JSON file for fluentd + OpenTelemetry), and pulling all the logs into Dagster seemed like the simplest way. I tried

helm show values

again and what I set didn't show up, so I think the issue may be with my helm calls (totally new territory for me).

if you want to see the values of your current Helm release, you should do

helm get values user-cloud -n dagster-cloud

where

user-cloud

is the name of your release, and

dagster-cloud

is your namespace

Found a typo converting from

lower_under

case that is in

dagster.yaml

to the

camelCase

that's in the Helm chart. Now It's picking up my log customization. Thanks for all the help!