Hello everybody! I'm new to dagster and I've been ...
# announcements
Hello everybody! I'm new to dagster and I've been running into some issues trying to get pipelines to execute correctly in Dagit. The first solid will run and finish, but then nothing else happens until the process times out. I recreated the "complex_pipeline" from the tutorial (which I was able to get working in dagit on my personal machine) and the behavior was the same. If I run either pipeline from the command line, it executes without an issue.
Below is a screenshot of a section that is different between the dagit and command line executions. This is consistent between the (2) pipelines I tried
Any help is appreciated 🙂
This machine is a Windows 10 w/python 3.7.3
I was able to get it to work by creating a venv in the same directory and installing all of the packages I needed (used Pycharm to create the project, which creates the environment in a common folder). Is it best practice to keep the environment in the same directory, or is there a way to fix this through configuraiton?
Hi David! Trying to understand more about your setup to repro that error. Were you invoking both the
command and
in the same directory? What tool are you using to create your venv? Curious because we should be invoking the same python executable for this forked process (to manage compute log output), but for some reason it cannot find the installed dagster module…
I am in the same same directory for both commands. The env that didn't work was created when I created a new pycharm project:
The environment that did work I used venv to create a venv called venv in the project folder.
I think this is a bug in Python venv…. I found this open issue: https://bugs.python.org/issue38905
It looks like there might be a possible workaround using a
environment variable, but I need to do more research to see if this can be handled from within dagster or if it has to be set by each user.
Do you mind filing a ticket on Github with this issue? It’s not likely to be a quick fix and I’d like to make sure we can track it (and other Windows users running into this can find it).
Will do. Do you have a preferred format for submitting an issue?
I think there should be a pre-filled rubric when you create the issue.
🕺 1