Hi dagsterfans I have a sensor that kicks off many small job

Question

Hi dagsterfans I have a sensor that kicks off many small jobs that each take less then a minute to complete however it takes up to a minute for the dagster to realize that the job is finished Is there a way to speed that up I m using run monitoring using the k8s job executor one container per job

Roel Hogervorst · Answer

I noticed the termination of subprocesses I use the multiproces executor takes about a minute too

Roel Hogervorst · Answer

finished steps in process in 1m1 seconds can t we make that faster