Hey, is there a way to configure the concurrency i...
# ask-community
m
Hey, is there a way to configure the concurrency inside an job? The docs only talk about the concurrency jobs...
bot responded by community 1
z
you can limit the number of concurrent steps that are running across all ops in a job when using the multiprocess_executor. I think this can be achieved on a more fine-grained level using the celery executors as well (listed here ), but I'm not as familiar with those
m
Is it also possible when running it in ECS?
z
yes, your job running in an ECS container (launched with EcsRunLauncher) can be configured to use a variety of executors. The multiprocess executor is one of those possible configurations (also the default). The docs I linked in my first reply shows how you can configure it in your run config, and also notes about default parallelism and execution priority tagging
m
Thanks, I will take a look