Hi all! Looking for some tips on how to make sure that the dagster UI keeps running smoothly. Right now I have some jobs that produce a lot of logs (about 70k logs) and this seems to slow down the dagit webserver, as I am trying to view the logs, but it just keeps loading. Is there some configuration or some other parameters that I need to set in order to make it run smoothly? Do I need to change my logging (most logs are `AssetMaterialization`s)? Or was I too conservative in resources? (2 GB mem, 1 CPU). Thanks in advance!

Hi Jaap! Some questions for you: What version of dagster are you running? What are you using for your dagster storage (postgres/mysql/sqlite)? Can you confirm, each run of your job is emitting 1000s of asset materializations? Are you seeing that just loading that run view is taking a long time, or that if you load a run in one browser tab, all requests hitting the same dagit instance are taking a long time? Also wondering what things look like in terms of resource utilization, using a profiler.

Hi @prha! I was running dagster==0.15.0. I was using the postgres storage for it (kubernetes helm chart defaults). The job fans out and each fanout had 1000 AssetMaterializations (running analyses of many different locations. I will use a schedule to do regular updates in time). I am new to dagster, so I might be looking at an asset in the wrong way, let me know. Loading the run view is the thing I am reporting on. Other shorter / less complex runs are doing ok. For the last point, what exactly do you want me to profile?