Is there anyway to view memory usage information f...
# dagster-serverless
z
Is there anyway to view memory usage information from the runner? We’re getting a few process run OOM while running with isolated executions but are quite surprised by this (FWIW: These processes use around ~5 GB of RAM when running on our local machines, and we don’t expect them to get close to the 16GB limit, yet still running OOM). Outside of that, is there anyone rationale of what could be going on here? I understand the dagster runner probably has some overhead, but even if we only were able to use 2/3 of the memory that should be plenty).
j
There’s not currently although it’s something we’d like to add in the future. Do you have a recent job you’d like me to look at? I can look through what we have instrumented internally to see if anything stands out.
z
Hey Jordan, thanks for reaching out! Sorry for the delay on my end 🙂 An example job that we’re trying to develop that is failing from this is
47ebd799-264f-4fdf-847a-e156378563a8
. My current hunch is that these jobs may be runnable due to more swap space on our osx / m1 dev machines when compared to the cloud agent. However; I haven’t done a closer investigation into it. Do you think the swap space discrepancy is likely/possible?