https://dagster.io/ logo
n

Noah K

10/11/2020, 1:43 AM
Specifically re: using remote executors, how does the code for the pipeline end up in a place the executor can run it?
d

daniel

10/11/2020, 1:08 PM
Hi Noah - Dagster assumes that remote executors like celery workers have access to the pipeline code already - e.g if you identify your pipeline using a python module, that module also needs to be available in the python environments in which the celery workers are running.
n

Noah K

10/11/2020, 8:48 PM
👍 Eventually found the code paths via selectively raising errors 😄
2 Views