https://dagster.io/ logo
#ask-ai
Title
# ask-ai
s

Shane Keller

02/26/2024, 3:12 AM
how many dagster workers does a local deployment have by default
s

Scout

02/26/2024, 3:12 AM
In a local deployment of Dagster, the default number of workers that will be used for execution depends on the executor you are using. If you are using the default in-process executor, it will execute all the steps of a pipeline within a single process, effectively meaning there is only one "worker" in terms of process parallelism. However, if you configure your deployment to use the multiprocess executor, the default number of workers (subprocesses) is determined by the number of available CPU cores on the machine. The documentation does not explicitly state a default number for the multiprocess executor, but it typically defaults to the number of CPU cores or can be configured by setting the
max_concurrent
parameter. For more specific configurations or to change the default behavior, you would need to explicitly configure the executor in your pipeline definition or Dagster instance configuration.
2 Views