Hey, thanks for all your help in the migration process! I have a new issue with the schedules and would like to ask for your help again. My project structured as follows:

pipelines/
--app/
----twitter/
------some_module.py
----workspaces.yaml
----repository.py
--home/
Pipifile
run.sh
...

To run dagit I use a bash script that adds

pipelines/app

to the

PYTHONPATH

so that the modules inside

app

can be found. It used to work also with the scheduled executions. With the migration to 0.8 the schedules are failing silently. However looking into the

results

file that the scheduler generates i found this error (please, apologize me for the formatting)

{"__class__": "IPCErrorMessage", "message": null, "serializable_error_info": {"__class__": "SerializableErrorInfo", "cause": null, "cls_name": "ModuleNotFoundError", "message": "ModuleNotFoundError: No module named 'twitter'\n", "stack": ["  File \"/home/user/.local/share/virtualenvs/user-pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/serdes/ipc.py\", line 116, in ipc_write_stream\n    yield FileBasedWriteStream(file_path)\n", "  File \"/home/user/.local/share/virtualenvs/user-pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/cli/api.py\", line 492, in launch_scheduled_execution\n    repo_def = recon_repo.get_definition()\n", "  File \"/home/user/.local/share/virtualenvs/user-pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/core/definitions/reconstructable.py\", line 36, in get_definition\n    return repository_def_from_pointer(self.pointer)\n", "  File \"/home/user/.local/share/virtualenvs/user-pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/core/definitions/reconstructable.py\", line 311, in repository_def_from_pointer\n    target = def_from_pointer(pointer)\n", "  File \"/home/user/.local/share/virtualenvs/user-pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/core/definitions/reconstructable.py\", line 271, in def_from_pointer\n    target = pointer.load_target()\n", "  File \"/home/user/.local/share/virtualenvs/user-pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/core/code_pointer.py\", line 84, in load_target\n    module = load_python_file(self.python_file)\n", "  File \"/home/user/.local/share/virtualenvs/user-pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/core/code_pointer.py\", line 73, in load_python_file\n    return import_module_from_path(module_name, python_file)\n", "  File \"/home/user/.local/share/virtualenvs/user-pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/seven/__init__.py\", line 82, in import_module_from_path\n    spec.loader.exec_module(module)\n", "  File \"<frozen importlib._bootstrap_external>\", line 783, in exec_module\n", "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n", "  File \"/home/user/user-pipelines/app/repository.py\", line 2, in <module>\n    from twitter.pipelines import (\n"]}}
{"__class__": "IPCEndMessage"}

It seems that the modules are not being picked well. Do you have advise on how, and where to setup the pythonpath so that the imports work? I guess this has to do with the new execution architecture?

To run dagit I use a bash script that adds pipelines/app to the PYTHONPATH

The issue is that

cron

is not going through that path

Hm, I see. However, why did it work on the 0.7.x? Also, where can I put this piece of code so it executes before the repository.py? as it already imports many modules

However, why did it work on the 0.7.x?

we used to manipulate the path which worked well in your case but caused problems for others, it also is more problematic in the new architecture that supports multiple repositories

I understand how that may fail for others

ya the other thing you could do is create a proper package and

pip install -e

it in to the python environment