https://dagster.io/ logo
#ask-community
Title
# ask-community
j

jan

04/26/2022, 3:08 PM
Hi there, I am confused as to why my usage of modules seems to not be working. I’m trying to get going with
dagster-airbyte
following this very brief tutorial by Aitbyte here. Steps: 1. fresh virtualenv, pip install dagster, dagit, dagster-airbyte 2. create a airbyte-dagster.py file with the tutorial contents in the same dir as the virtualenv dir 3. run
dagit -f airbyte_dagster.py
4. This results in the following error and leaves me confused
Copy code
0it [00:00, ?it/s]
0it [00:00, ?it/s]
/opt/homebrew/lib/python3.9/site-packages/dagster/core/workspace/context.py:558: UserWarning: Error loading repository location airbyte_dagster.py:dagster.core.errors.DagsterImportError: Encountered ImportError: `No module named 'dagster_airbyte'` while importing module airbyte_dagster from file /Users/jankyri/git/kmo/local-dagster-dbt-meltano/airbyte_dagster.py. Local modules were resolved using the working directory `/Users/jankyri/git/kmo/local-dagster-dbt-meltano`. If another working directory should be used, please explicitly specify the appropriate path using the `-d` or `--working-directory` for CLI based targets or the `working_directory` configuration option for `python_file`-based workspace targets. 

Stack Trace:
  File "/opt/homebrew/lib/python3.9/site-packages/dagster/grpc/server.py", line 209, in __init__
    self._loaded_repositories = LoadedRepositories(
  File "/opt/homebrew/lib/python3.9/site-packages/dagster/grpc/server.py", line 94, in __init__
    loadable_targets = get_loadable_targets(
  File "/opt/homebrew/lib/python3.9/site-packages/dagster/grpc/utils.py", line 33, in get_loadable_targets
    else loadable_targets_from_python_file(python_file, working_directory)
  File "/opt/homebrew/lib/python3.9/site-packages/dagster/core/workspace/autodiscovery.py", line 24, in loadable_targets_from_python_file
    loaded_module = load_python_file(python_file, working_directory)
  File "/opt/homebrew/lib/python3.9/site-packages/dagster/core/code_pointer.py", line 100, in load_python_file
    raise DagsterImportError(

The above exception was caused by the following exception:
ModuleNotFoundError: No module named 'dagster_airbyte'

Stack Trace:
  File "/opt/homebrew/lib/python3.9/site-packages/dagster/core/code_pointer.py", line 86, in load_python_file
    return import_module_from_path(module_name, python_file)
  File "/opt/homebrew/lib/python3.9/site-packages/dagster/seven/__init__.py", line 47, in import_module_from_path
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "airbyte_dagster.py", line 2, in <module>
    from dagster_airbyte import airbyte_resource, airbyte_sync_op

  warnings.warn(
a

alex

04/26/2022, 3:13 PM
ModuleNotFoundError: No module named ‘dagster_airbyte’
can you do a
pip list | grep dagster
?
j

jan

04/26/2022, 3:13 PM
Sure thing!
Copy code
dagster                0.14.12
dagster-airbyte        0.14.12
dagster-graphql        0.14.12
dagster-shell          0.14.12
a

alex

04/26/2022, 3:15 PM
puzzling indeed
in the same dir as the virtualenv dir
how are you managing your virtual envs?
j

jan

04/26/2022, 3:18 PM
Python’s
virtualenv
package.
I.e.
virtualenv venv
,
source ven/bin/activate
,
pip install foobar
Same thing with
python3 -m venv fooenv
.
a

alex

04/26/2022, 3:22 PM
and you put this tutorial code in the same directory? I think we may be making assumptions that are incompatible with the code being in the exact same folder as the venv.
j

jan

04/26/2022, 3:24 PM
Yeah. Oh, I did that after having it in a different dir. So
Copy code
-- dagster_pipelines
|  |
|  |-- airbyte_dagster.py
|
|- venv/
was what I had before. Same outcome. EDIT: @alex different dir works now. Aaaaaargh 😵
So as in the EDIT, it works now. @alex following up on my second question though, is there any docs I am not aware of which transparently give instructions on the process of how modules are discovered and what’s required in terms of setup from a user?
a

alex

04/26/2022, 3:29 PM
i think this is probably the best thing we have in that regard https://docs.dagster.io/concepts/repositories-workspaces/workspaces#workspaces
👍 1
j

jan

04/26/2022, 3:32 PM
Great, ty!