clay
02/09/2023, 10:45 PMRUN
line at the end:
CMD ["dagster", "api", "grpc", "-h", "0.0.0.0", "-p", "4000", "--max_workers", "3", "--module-name", "my_module"]
If I materialize all of the assets in a job, dagster spins up 3 workers to handle concurrent processing. However, once it is all complete, if I again materialize all of the assets in the job (as I was doing as part of testing), dagster only spins up 2 workers (which is default). Is that expected or should I be specifying --max_workers
elsewhere, too?daniel
02/09/2023, 11:09 PMdaniel
02/09/2023, 11:09 PMclay
02/09/2023, 11:09 PMclay
02/09/2023, 11:14 PMmax_concurrent
in the job will determine how many code location containers are launched to complete the work?daniel
02/09/2023, 11:14 PMdaniel
02/09/2023, 11:14 PMdaniel
02/09/2023, 11:15 PMclay
02/09/2023, 11:15 PM