Hey everyone, I am running into a weird ModuleNotFound error when launching a workflow in a docker_user_code container. My code lives at

/app/module

inside the docker container, and I am able to successfully run

python -m module.pipeline

inside it, and the user_code rpc server is successfully able to find the module, but the Dagster execution fails. I have tried setting

-d

and

--working-directory

to

/app

but neither of them help, any ideas?

Hi Roshan - would you mind posting the full text of the error?

Yup! The line is

zeus.jobs.dagster_repo

<- this module contains 2x repos, and its called via

dagster api -d /app -m zeus.jobs.dagster_repo

dagster.core.errors.DagsterImportError: Encountered ImportError: `No module named 'zeus'` while importing module zeus.jobs.dagster_repo. Local modules were resolved using the working directory `/app`. If another working directory should be used, please explicitly specify the appropriate path using the `-d` or `--working-directory` for CLI based targets or the `working_directory` configuration option for workspace targets.
  File "/usr/local/lib/python3.9/site-packages/dagster/grpc/impl.py", line 77, in core_execute_run
    recon_pipeline.get_definition()
  File "/usr/local/lib/python3.9/site-packages/dagster/core/definitions/reconstructable.py", line 150, in get_definition
    defn = self.repository.get_definition().get_pipeline(self.pipeline_name)
  File "/usr/local/lib/python3.9/site-packages/dagster/core/definitions/reconstructable.py", line 72, in get_definition
    return repository_def_from_pointer(self.pointer)
  File "/usr/local/lib/python3.9/site-packages/dagster/core/definitions/reconstructable.py", line 605, in repository_def_from_pointer
    target = def_from_pointer(pointer)
  File "/usr/local/lib/python3.9/site-packages/dagster/core/definitions/reconstructable.py", line 534, in def_from_pointer
    target = pointer.load_target()
  File "/usr/local/lib/python3.9/site-packages/dagster/core/code_pointer.py", line 194, in load_target
    module = load_python_module(self.module, self.working_directory)
  File "/usr/local/lib/python3.9/site-packages/dagster/core/code_pointer.py", line 130, in load_python_module
    raise DagsterImportError(
The above exception was caused by the following exception:
ModuleNotFoundError: No module named 'zeus'
  File "/usr/local/lib/python3.9/site-packages/dagster/core/code_pointer.py", line 125, in load_python_module
    return importlib.import_module(module_name)
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked

oh, one thing that is possibly confusing here - this import error could be the result of another import statement within the code, it's not neccesarily specifically the result of being able to not import the 'zeus.jobs.dagster_repo' module

I just skimmed through my codebase and couldn't find any other place where this import exists, only in the Dockerfile for launching the user_code container

would you mind sharing your workspace.yaml file?

Yup! I set it in the docker compose

cool, and the image/tag of the run container is exactly the same as the image of the gRPC server?

Yes. I am just developing locally, and rebuild before each deployment of the grpc server. Can you give me a bit? I am off my computer atm, but I'll send the command here asap

sure, no rush

Oh, that's my bad - poor choice of words. I meant that I tried running the code locally without docker/the gRPC server and I was able to execute it. The gRPC server itself is up and running successfully.

If you run docker inspect on the two containers (the gRPC server and the container that the run launcher spins up) are there any differences? For example, the code in the gRPC server isn't mounted a a volume or anything like that by any chance?

I can take a look at the container that the run launcher spins up, the gRPC server has the correct mount though. Sec

Oh, is the code mounted as a volume? That would explain everything if it is

Oh yeah it is! Could you point me to an example?

https://docs.dagster.io/deployment/guides/docker#mounting-volumes this has example config