Hi Guys! Can I control the multiprocessor executor count on individual pipelines or using tags? We have different memory requirements on pipelines and some fail with ChildProcessCrashed and others need a high amount of concurrent processes.

Hi Sebastian, you can use the configured API to set this configuration on individual pipelines in code: https://docs.dagster.io/concepts/configuration/configured#configured-api

Oh great, how did I miss that. I thought it's only working in modes. Thanks!

i'd expect something like this to work (pseudocode)

from dagster import multiprocess_executor

configured_executor = multiprocess_executor.configured({...}) # your config here

@job(executor_def=configured_executor)

Yeah, I was looking for something like this, but here this attribute is "missing"

oh, the example I gave was with the new graph/job APIs

I see, that's how I was using it. So I'd have to create a different mode or move the code to the new API. Thanks for the help.