Hey everyone, I am having issues trying to get for...
# ask-community
j
Hey everyone, I am having issues trying to get formatted python logs to write to standard out, but I am experiencing some issues. I am using the DockerRunLauncher, however despite my python logger being configured properly as a managed python logger and all logging statements showing up in the dagit UI, it does not seem like any of the logging statements are being written to standard out inside the Docker container spawned for the process. Has anyone experienced an issue similar to this using the DockerRunLauncher, or have an example of how to write to standard out within the process container?
o
hi @Jim Nisivoccia -- I believe this is a quirk of how the
docker logs
command works. It only reads stdout from the process with PID=1, and the default executor for jobs will run each step in a different process (so stdout from those steps will not be visible when you run
docker logs
). If you don't mind the potential for a little bit of duplication in your log output, the easiest solution is likely to update your dagster.yaml file to look something like this:
Copy code
python_logs:
  managed_python_loggers:
    - mylogger
  dagster_handler_config:
    handlers:
      myHandler:
        class: logging.FileHandler
        level: INFO
        filename: "/proc/1/fd/1"
(this will tell dagster to write all logs to the stdout of the process with PID=1, which
docker logs
will pick up).
just for my understanding, are you looking directly at the docker logs for debugging purposes, or are you trying to feed them to an external system?
j
Thanks for the helpful response @owen! I will try this out and let you know what happens. We have an elasticsearch cluster picking up logs from multiple containers on an EC2 instance, and these dagster process containers are being spawned on the same instance so I figured we should be able to see their logs in elastic. When checking these process container's logs using docker logs, they are empty. Once these containers begin to log to std out, then we should see those logs also appear with docker logs and in elastic
This solution worked for me, thank you for your help!
o
no problem!