Hi, is there a way to change/configure the Python ...
# announcements
o
Hi, is there a way to change/configure the Python venv that a whole pipeline and individual solids are run under?
Hmm, I guess I could use the bash solid. Not sure that's the best way though.
a
not currently (from dagit at least) - how are you starting these executions? if you use the python APIs or dagster / dagster-graphql CLIs you could execute a pipeline from within the specififc venv
m
could write a run launcher that did this
👍 1
o
I am currently launching the executions from within dagit. I am looking for a way to enable folk to work on different development streams of the same job, so launch in different venvs. I guess, it could be done via API, but then we lose all the benefits of the UI.
A complexity of this is that I might want to run different solids in different venvs. E.g. use current prod versions for parts A,B and D of my pipeline, but a dev venv for part C.
a
makes sense - this is something we expect to make easier / possible in the future but haven’t quite reached yet. It would be possible to write a custom
RunLauncher
and add it to your instance and that run launcher could use a
tag
on the pipeline to choose a venv to use to execute
dagster-graphql
in to satisfy the execution request