Is there a way to run different jobs using different packages / package versions? We have code that currently requires different python virtual environments. We're currently using a Serverless deployment

Hi Ryan, the best way to set up a different python environment for certain jobs is to define them in a different code location. If you’re working from one of the quickstart templates, you can add another entry to the

dagster_cloud.yaml

file in order to build/deploy another code location.

And where do I define the requirements? I'm currently adding them to the

setup.py

file in my project root

Hi Ryan, to use different requirements per job you will need to create a sub directory per code location, each with its own

setup.py

. Then in your

dagster_cloud.yaml

you specify the directory for each code location by

build: directory:

. For example, with this structure

project/
  subdir1/
    module1/
    setup.py
  subdir2
    module2/
    setup.py
  dagster_cloud.yaml

your

dagster_cloud.yaml

would contain

locations:
  - location_name: subdir1
    code_source:
      package_name: module1
    build:
      directory: subdir1
  - location_name: subdir2
    code_source:
      package_name: module2
    build:
      directory: subdir2

To share code across these directories you can add another directory next to

subdir1

, eg

shared3

and then add the following in

subdir1/requirements.txt

and `subdir2/requirements.txt`:

../shared3

awesome, thanks!