Hello, I'm running into a `ModuleNotFoundError: No...
# announcements
b
Hello, I'm running into a
ModuleNotFoundError: No module named '<my_module>'
error when I try to run Dagit. I recently upgraded my project to use the
workspace.yaml
structure. I can provide more details but does anyone have an idea what may be causing the issue?
d
hi Basil - I bet we can sort this out, could you paste the contents of your workspace.yaml?
b
Thanks for the help! The structure of my project is basically:
Copy code
top_level_project_folder
    my_module
        __init__.py
        repo.py
    workspace.yaml
and my workspace.yaml file contains:
Copy code
load_from:
  - python_module:
      module_name: my_module.repo
      attribute: my_repository
I'm running
dagit
from the same directory as
workspace.yaml
. I've tried importing/running
find_packages
from
setuptools
in the same directory and
my_module
shows up.
And
my_repository
is a function in my_module/repo.py decorated with @repository
Also a related question I've seen both
python_module
and
python_package
used in the
workspace.yaml
file—which one is correct? (currently tried both and getting same error)
One other thing to note—I'm running
dagit
from inside a pipenv shell
a
cc @prha we default the working directory to the location of
workspace.yaml
so since you depend on working directory to load the module, you may be best off using
python_file
instead of
python_module
p
that’s right…. both
python_package
and
python_module
expect the specified package to be installed
b
When I use
python_file
I get this error:
Copy code
ImportError: `attempted relative import with no known parent package` while importing module repo from file /path/to/my_module/repo.py. Consider using the module-based options `-m` for CLI-based targets or the `python_package` workspace.yaml target.
and
workspace.yaml
in this case:
Copy code
load_from:
  - python_file:
      relative_path: my_module/repo.py
      attribute: my_repository
I don't understand where I'm going wrong I feel like I've paralleled the examples in your docs pretty closely.
Appreciate the help!
p
sure, yes, this is kind of an annoying issue that we haven’t found a good workaround for
python doesn’t like relative imports in non-packages, but we need to be able to reproducibly load your code
is it possible to do a local installation of your module / package?
b
Yeah its possible but does that mean I need to reinstall locally every time I change my package? There isn't a development workflow where I can run dagit locally?
I was able to do this before same project structure when you guys used the
repository.yaml
file workflow
p
We use
pip install -e path/to/module
to do an editable install for local development
yeah, it’s a headache, but before workspaces, we were running dagster code and user-provided pipeline logic code all together
doing this code separation cleaned up a lot to separate out those environments, but caused some code loading annoyances like
attempted relative import with no known parent package
errors
b
So in my case I'd run
pipenv install -e path/to/module
so I have my own module as a dependency for my project? How does that translate when I move things to production?
p
We may try to find another workaround for
0.10.0
to resolve these headaches, but any change to code loading results in some breaking changes so we’re reluctant to do that in one of the weekly releases.
i’m not sure what your production deployment process is, but you would still check out your code as it currently is
b
Ok. No problem. I think that solved the issue for now. Thanks for your help and explaining things. I'll keep my eye out for changes in
0.10.0
Thanks!
290 Views