The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi, what are the options for executing solids in a pipeline in parallel? Is it possible to declare sets of solids which should be executed in parallel?

hi richard, some executors support parallel execution and some don't -- so, for instance, by default you can use the multiprocessing executor by adding a chunk of config like
```execution:
  multiprocess:```

the executors will determine what can be executed in parallel by looking at the data dependencies between solids

so any solids whose dependencies are satisfied can be executed in parallel

if you need to explicitly sequence solids but can't pass data between them for some reason, you can use inputs/outputs with the type `Nothing`

Thanks max that makes sense. Just reading through the Executor docs now. We have solids that might look like they can run parallel but actually need to be run in sequence. Going by the docs I think we could write an executor that would suit our needs.