Hi everyone, we're running Dagster on Fargate. In...
# deployment-ecs
Hi everyone, we're running Dagster on Fargate. In the pipeline below, after the
op, there are a bunch of ops generated using
. All these ops could run in parallel. But for some reason, it seems they only run two at a time. Does anyone know which setting I can tweak to increase the number of parallel tasks?
I suppose this is configured here, but I'm not 100% sure: https://docs.dagster.io/_apidocs/execution#dagster.multiprocess_executor
That was it, @Grigoriy Sterin! Thank you for the tip!
I'm setting up to do something similar, so curious to understand what you did better. Are you running one op per container, or one job per container?