https://dagster.io/ logo
#dagster-plus
Title
# dagster-plus
r

Ryan Meekins

03/27/2023, 3:33 PM
Is there a way to run different jobs using different packages / package versions? We have code that currently requires different python virtual environments. We're currently using a Serverless deployment
b

ben

03/27/2023, 3:42 PM
Hi Ryan, the best way to set up a different python environment for certain jobs is to define them in a different code location. If you’re working from one of the quickstart templates, you can add another entry to the
dagster_cloud.yaml
file in order to build/deploy another code location.
r

Ryan Meekins

03/27/2023, 3:45 PM
And where do I define the requirements? I'm currently adding them to the
setup.py
file in my project root
s

Shalabh Chaturvedi

03/27/2023, 4:46 PM
Hi Ryan, to use different requirements per job you will need to create a sub directory per code location, each with its own
setup.py
. Then in your
dagster_cloud.yaml
you specify the directory for each code location by
build: directory:
. For example, with this structure
Copy code
project/
  subdir1/
    module1/
    setup.py
  subdir2
    module2/
    setup.py
  dagster_cloud.yaml
your
dagster_cloud.yaml
would contain
Copy code
locations:
  - location_name: subdir1
    code_source:
      package_name: module1
    build:
      directory: subdir1
  - location_name: subdir2
    code_source:
      package_name: module2
    build:
      directory: subdir2
To share code across these directories you can add another directory next to
subdir1
, eg
shared3
and then add the following in
subdir1/requirements.txt
and `subdir2/requirements.txt`:
Copy code
../shared3
r

Ryan Meekins

03/27/2023, 8:20 PM
awesome, thanks!
4 Views