Hello! I have an existing Dagster project using `...
# announcements
n
Hello! I have an existing Dagster project using
v0.7.13
and I'm currently trying to update it into
v0.8.5
. I'm encountering an error where the scheduler for
v0.8.5
can't find my
pipelines
directory. For context, here's how
v0.7.13
was structured:
Copy code
dagster
- solids
    - test.py
- pipelines
    - test.py
- scheduler.py
- repository.py
- repository.yaml
- dagster.yaml
And here's how I structured
v0.8.5
:
Copy code
dagster
- solids
    - test.py
- pipelines
    - test.py
- repository.py
- dagster.yaml
- workspace.yaml
Everything's working fine except for the scheduled run which is raising
ModuleNotFoundError: No module named 'pipelines'
. It can access the
pipelines
folder when ran manually (using dagster run and by running the generated script) but not when scheduled. Added a simplified version of my
repository.py
here: https://gist.github.com/santosnarom/7f42ba94d29fb8395f06ae2b12b8cb85
a
The issue is that that relative import only works when the
current working directory
is your folder, and when dagster executes from cron it does not change directories. You can: * make it a proper
package
and
pip install -e
it. * add the directory to the python path with some shenanigans like https://github.com/dagster-io/dagster/blob/master/.buildkite/step_builder.py#L7-L9
cc @prha who was looking in to providing better support for this
p
Yes, I’m actively working on addressing this issue. Hoping to get something out in the next release. Created an issue to track this: https://github.com/dagster-io/dagster/issues/2664
n
Thank you!