Is there a way to configure logging as JSON logging globally

Question

Is there a way to configure logging as JSON logging globally we have Kubernetes deployment and NewRelic log ingestion at cluster level I do not want to touch the code is there a way to switch the logger formats and loggers to JSON formatted logging only

ben · Answer

Hi Binoy this may not match exactly what you re hoping for but you can specify a logger in a singular place e g once per code location by attaching it to the `Definitions` object <https docs dagster io concepts logging loggers specifying default code location loggers> You d have to modify the code but only in one place

Binoy Shah · Answer

Hi Ben thanks for the link I have seen it While I love the framework logging feels a bit patchy and too many knobs kind of affair

ben · Answer

What would your ideal logging setup look like This is something we ve been talking about improving internally so any feedback would be very useful for us

Binoy Shah · Answer

It would be rather great to see multiple global loggers configured with `dagster yml` or some sort of `logger yaml` that controls various default loggers ```dagit logger logger class dagster logging ConsoleLogger logger format xxx log level INFO enable color true daemon logger logger class dagster logging JSONLogger combine stacktrace true log level ERROR job logger logger class dagster logging FancyTracingLogger log level DEBUG log format xxx ```

Binoy Shah · Answer

This file if it s relied upon then it can be controlled by kubernetes Helm values too and each of them can internally be associated with some kind of optional persistent backend Same works within docker or overridden by a volume mount and locally too Dagster project generator dumps out a starter `logger yaml` file with commented loggers and dagster dagit start with default values

Binoy Shah · Answer

So if this is sample of builtin logger <https docs dagster io concepts logging loggers configuring built in loggers> Questions 1 how do I override to use json formatting 2 How do find what other loggers and config values are allowed used 3 documentation shows a single entry under heading of Configuring built in loggers Where are additional configuration fields Tip Here It has a statement add the following snippet to your config YAML why does nt it say what is that config yaml is it `dagster yaml` or `workspace yaml` it takes a mental mapping to figure out which ` yaml` file this goes under

Andrea Giardini · Answer

point up Good point made here and something that I am facing as well I think there are certain settings like Json logging that make sense to have instance wide rather than per repo per definition