Hi All! I’m running into postgresql timeout in Dagit for a job with a large amount of logs. I’ve gone through custom Dagster loggers but as I understand, they apply to what gets logged on the terminal and not Dagit? additionally, is there a simpler way to just increase the postgresql timeout?

dagit

has a cli flag

--db-statement-timeout INTEGER  The timeout in milliseconds to set on
                                  database statements sent to the
                                  DagsterInstance. Not respected in all
                                  configurations.  [default: 15000]

Thanks for the input @alex! Is there a way to reduce the amount of logs show in Dagit UI? As I understand custom loggers only apply to the terminal, will be great if we can have some options to customise the amount of logs in the Dagit UI too!

No specific controls for reducing system events at this time. You can consolidate work in to fewer

ops

to reduce the per-op overhead of events. Also make sure you are minimizing use of

context.log

We have thousands of Ops that run in parallel. The large number of logs are largely due to them