Hi, I found this in the docs but couldn’t find how to actually achieve that (https://docs.dagster.io/concepts/code-locations) - do you have any pointers?

Each code location can be sourced from a separate Python environment, so teams can manage their dependencies (or even their Python versions) separately.

Here’s a pointer to a tutorial - there are likely others, this is just the first one I found https://realpython.com/python-virtual-environments-a-primer/

So my question is specifically on having different dagster code location run on the back of a different environment

https://docs.dagster.io/concepts/code-locations/workspace-files#loading-multiple-repositories here’s an example using a workspace file that does this

I saw that - but I don’t see where you can specify the virtual env to run each location on?

Each virtual environment has its own python executable path, so the executable_path key determines the virtual environment

Hi @daniel, figured it would be worth tagging you in this thread since it looks like Marco was having the same problem I'm having. I did indeed (after a lot of searching) find the snippet you linked on my own, however I'm left with two concerns: 1. The section you linked notes that "This is applicable only for code locations defined using

@repository

." Is that accurate? Seems like much of this section probably applies to Definitions as well. 2. I feel like I'm piecing together random bits from various different parts of the docs regarding workspaces - for example: location_name and executable_path are only mentioned on this page (which is the Workspace Files docs page) once I click the tab you linked for "Loading multiple repositories," which I initially did NOT click because my thought was "well this isn't relevant to me since I don't plan to use repositories since the docs have been telling me to use the more recent Definitions concept." My point is I'm having a really hard time figuring out how to do something that Dagster claims it is good at: isolating dependencies across projects/teams. So: a. Do you know of any place that has more verbose docs on workspaces and all the possible fields I can specify? Or any really good examples I can parse through? b. If I may suggest - it feels like this is a blind spot in the docs. Surely I'm not the first person who was using Airflow, has had incredible headaches with dependencies, saw that Dagster should resolve them - and then struggled to find out how to do so in the docs. I might even argue that this topic (how to set up Dagster to handle many different dependency requirements) deserves its own docs page. For example: I assume I also need to define those venvs in my build somehow? Such a thing + an example feels worthy of docs.

Hey Conor - I think this is really great feedback, any chance you could post the second part in #dagster-feedback to make sure that the right folks on the team (in particular the people thinking about holes in our docs) see it?

Yes of course! Do you have any suggestions for me, or will folks over there be better able to help?

I think you're correct that loading multiple repositories (should probably say "loading multiple code locations" is just as applicable for Definitions as it is for repositories)

Right I plan to do some kind of dockerized deployment, although our infra people may ultimately involve kubernetes I assume. But point being I need to ensure all the right venvs exist on my own outside of dagster's configs right? As in, Dagster doesn't deal with creating or maintaining those.

so if you're doing docker or kubernetes - another option that achieves a similar goal is to make a different Docker image for each code location

Ok yeah I figured, makes enough sense, thanks!

Thanks for doing this update, the examples are helpful. I was just about to ask the same question about how to configure those python envs, while looking at the updated page (had the question "how do I do that?" before I finished reading the page and went off to search) - maybe including an executable_path in one of the examples in the "configuring code locations" section would make it more obvious.

Thanks Daniel. I was also able to set multiple code location based on the docs, in the end. One problem I have at the moment though is how to set up env variables prior to the ‘executable path’ relative to a given code location being executed. Or more generally running a ‘custom’ executable, e.g. a script that sets env vars prior to calling Python. If I do that I get a ‘[Errno 8] Exec format error’. Any idea how to solve?

Would it be possible to make a new post for that question? That way our support oncall will see it

👍🏻