12/14/2022, 5:54 PM
Hello! We are using dagster-celery to distribute jobs out at the op level by the queue tag. Most of the ops have non-python dependencies (e.g. external tools/executables with varying requirements and dependencies of their own) that we have isolated to separate docker images running the workers. Is there a way to define sort of an interface (possibly similar to a celery signature) for an asset/op/etc. providing the typed inputs and outputs, but having the implementation isolated to the celery worker codebase? According to the docs, the code has to be available in both dagit and the workers. Does the actual compute_fn of an op need to be present in dagit when it's the celery worker executing it?