Hi Guys! Can I control the multiprocessor executo...
# ask-community
Hi Guys! Can I control the multiprocessor executor count on individual pipelines or using tags? We have different memory requirements on pipelines and some fail with ChildProcessCrashed and others need a high amount of concurrent processes.
Hi Sebastian, you can use the configured API to set this configuration on individual pipelines in code: https://docs.dagster.io/concepts/configuration/configured#configured-api
Oh great, how did I miss that. I thought it's only working in modes. Thanks!
@daniel Could you show example code on how to set up the decorator (I suppose)? It's not obvious to me where I need to plug in the configured executor
i'd expect something like this to work (pseudocode)
Copy code
from dagster import multiprocess_executor

configured_executor = multiprocess_executor.configured({...}) # your config here

Yeah, I was looking for something like this, but here this attribute is "missing"
oh, the example I gave was with the new graph/job APIs
in the old APIs, the executor def would go on the mode I believe
ModeDefinition takes in a list of ExecutorDefinitions in its constructor
I see, that's how I was using it. So I'd have to create a different mode or move the code to the new API. Thanks for the help.
condagster 1