hey random question feedback about docs there are a few plac

Question

hey random question feedback about docs there are a few places in the documentation where it says update your config yaml but it doesn t say which config yaml and it also doesn t say which section <https dagster readthedocs io en 0 6 6 sections learn guides logging logging html configuring the built in loggers> i assumed it meant `dagster yaml` but i get `Undefined field loggers at the root Expected ` so i m not sure where to put this

prha · Answer

Hey Chris

prha · Answer

The loggers get configured on a per execution basis

Chris Roth · Answer

gotcha

prha · Answer

so these would get set in the config editor e g playground

prha · Answer

But thanks for raising that this is confusing we ll need to find a better way of distinguishing all of our config

Chris Roth · Answer

that makes sense i guess when it says config yaml i should hae thought of that

prha · Answer

probably something like pipeline config vs instance config

Chris Roth · Answer

i feel like i m never sure which config it s referring to since there are so many yaml configs

Chris Roth · Answer

since there is `repository yaml` `dagster yaml` the playground execution yaml `celery yaml` `scheduler yaml`

Chris Roth · Answer

the issue i m trying to debug actually is that when i click view raw step output on steps that were run on celery i just get No log file available any idea

prha · Answer

Do you have a `compute logs` section in your instance yaml `dagster yaml`

prha · Answer

also are your celery workers on a remote box

prha · Answer

You might need to configure a `ComputeLogManager` that stores the compute logs remotely check the `dagster aws` package for the `S3ComputeLogManager`

prha · Answer

Your `dagster yaml` would then include a section like this ```compute logs module dagster aws s3 compute log manager class S3ComputeLogManager config bucket my bucket prefix my prefix ```

Chris Roth · Answer

yup here is my section ``````

Chris Roth · Answer

```compute logs module dagster aws s3 compute log manager class S3ComputeLogManager config bucket myp pipeline prefix logs```

Chris Roth · Answer

i m having the issue both on our production ECS cluster and on my local docker with dagit + redis worker setup

prha · Answer

hmm

Chris Roth · Answer

s3 is otherwise working though

Chris Roth · Answer

i m calling `context resources s3 download file` which is using `dagster aws s3 resources` for example and it works correctly

prha · Answer

taking a look

Chris Roth · Answer

thank you

prha · Answer

did this used to work and was a regression or are you trying to set this up for the first time

Chris Roth · Answer

i m not sure tbh it s been set up for a month or two already but i haven t really tried to access the stdout stderr feature in production until now

prha · Answer

and you see No log file available for in flight steps or also for completed steps

prha · Answer

It s a known issue that the logs do not currently stream as the step is executing

Chris Roth · Answer

completed

prha · Answer

Can you check if the compute logs did get written to the bucket

Chris Roth · Answer

it s writing the log files but they are empty