For `start_method: forkserver` would it make sense...
# dagster-feedback
m
For
start_method: forkserver
would it make sense to always preload the code location? For example if I run
dagit -m orchestration.main
=> always have
orchestration.main
in
preload_modules
. From advice in https://dagster.slack.com/archives/C01U954MEER/p1647575140840359 I played around with preloading different modules, and I get a big effect (1 minute => 23s per job run) by preloading my main module. Seems logical, since (a) it's always used, and (b) it indirectly imports all the other modules.
c
that's a pretty drastic performance increase... made an issue to track. https://github.com/dagster-io/dagster/issues/8638. The person who has been mainly working on our multiprocessing forkserver implementation is on PTO right now, so might not get addressed right away, but I suspect that it might have been just an oversight in the original implementation.
ty thankyou 1