Is there a way to inspect the number of executors ...
# ask-community
m
Is there a way to inspect the number of executors a job is running with (within the job itself)?
OpExecutionContext.run_config
allows us to inspect the run configurations, and I'm able to inspect
max_concurrent
in there, but only if it's provided as a value when launching the job via Dagit.
🤖 1
based on this:
Copy code
The max_concurrent arg is optional and tells the execution engine how many processes may run concurrently. By default, or if you set max_concurrent to be 0, this is the return value of python:multiprocessing.cpu_count().
so, i can inspect the
OpExecutionContext.run_config
, get the
execution.config.max_concurrent
key. if it's 0 or doesn't exist as a key in the
run_config
, then i should just use the
multiprocessing.cpu_count()
value? is there a built-in alternative?
s
There is no built-in alternative, but this is something I think we should add to the
OpExecutionContext
. Would you mind opening a GH issue?
❤️ 1
m
will do, thanks!
🙏 1