The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi, we’re using the `dagster api grpc-health-check`  command as our kubernetes livenessProbe for our pipelines container (every 5 seconds). This is causing significant load on the CPU. Aside from reducing the frequency of the liveness probe, are there any other fixes for this/plans to streamline this in a future release? We’re running 0.10.0

let me do some profiling

we did change the default threadpool size in `0.10.3` though I doubt there is a busy-wait if the thread pool is under resourced

can you provide some more context on what exactly you are observing? Does the CPU just jump every 5 seconds?

most of the cpu time is in imports - was able to move some things around and cut it in half

Thanks <@UH3RM70A2> - the CPU only jumps every 5 seconds - it seems like your attribution to imports is right. We’ll probably reduce the livenessProbe frequency anyway.
The changes you made to the imports - will that be available in the next release?

yep they are landed should come out tomorrow