Hi, I just wanted to know if there is a way to run an op with parallel processing or so?
I need some parallel processing for some jobs and I thought that Dagster could have some integration with one of these libs which allow us to create heavy processes and parallel
07/18/2022, 3:48 PM
Hi @Ismael Rodrigues The way jobs are executed is controlled by executors. by default the multiprocess executor is used which will run ops in parallel processes if they are not dependent on each other in the graph. There are other executors available to you depending on how you are deploying your code (k8s executor, celery executor etc)
09/01/2022, 1:36 AM
Oh, thanks, by the way, I just forgot to answer how careless for my side I hope you're doing well :)