The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi Guys!

Can I control the *multiprocessor executor count* on individual pipelines or using tags?

We have different memory requirements on pipelines and some fail with ChildProcessCrashed and others need a high amount of concurrent processes.

Hi Sebastian, you can use the configured API to set this configuration on individual pipelines in code: <https://docs.dagster.io/concepts/configuration/configured#configured-api>

Oh great, how did I miss that. I thought it's only working in modes. Thanks!

<@U016C4E5CP8> Could you show example code on how to set up the decorator (I suppose)? It's not obvious to me where I need to plug in the configured executor

i'd expect something like this to work (pseudocode)
```from dagster import multiprocess_executor

configured_executor = multiprocess_executor.configured({...}) # your config here

@job(executor_def=configured_executor)```

Screenshot 2022-02-09 at 16.00.06.png

Yeah, I was looking for something like this, but here this attribute is "missing"

oh, the example I gave was with the new graph/job APIs

in the old APIs, the executor def would go on the mode I believe

ModeDefinition takes in a list of ExecutorDefinitions in its constructor

I see, that's how I was using it. So I'd have to create a different mode or move the code to the new API. Thanks for the help.