https://dagster.io/ logo
#ask-ai
Title
# ask-ai
s

Selim Raboudi

02/23/2024, 4:51 PM
@Scout say I have this folder structure: common/ pipelines/ pipeline_1/ pipeline_2/ dagster_cloud.yaml How should I write dagster_cloud so that it loads correctly all my code locations (defined by python packages pipeline_1/ and pipeline_2/
s

Scout

02/23/2024, 4:51 PM
Based on the documentation, the
dagster_cloud.yaml
file should be placed in the root of your Dagster project. If you have multiple Dagster projects in subdirectories, such as a monorepository, you should add the
dagster_cloud.yaml
file to the root of where the Dagster projects are stored, which you have done correctly. To load your code locations defined by the Python packages
pipeline_1
and
pipeline_2
, you would specify them in the
dagster_cloud.yaml
file using the
locations
key. Each location will have a
location_name
and a
code_source
that specifies how the code location is sourced. Since you are dealing with Python packages, you would use the
package_name
parameter within
code_source
. Here is an example of how you might structure your
dagster_cloud.yaml
file:
Copy code
yaml
locations:
  - location_name: pipeline_1_location
    code_source:
      package_name: pipeline_1
    working_directory: ./pipelines/pipeline_1

  - location_name: pipeline_2_location
    code_source:
      package_name: pipeline_2
    working_directory: ./pipelines/pipeline_2
In this example,
location_name
is a user-defined name for the code location that will appear in the Dagster UI. The
package_name
is the name of the package containing your Dagster code, and
working_directory
is the path of the directory that Dagster should load the code source from. Please adjust the
package_name
and
working_directory
as needed to match your actual package names and directory structure. If your
pipeline_1
and
pipeline_2
directories are structured as Python packages (i.e., they contain an
__init__.py
file), then the above configuration should correctly load your code locations. dagster_cloud.yaml reference | Dagster D... button Dagster project files | Dagster Docs button Dagster Databricks Step Launcher package... button Materialization of non-partitioned asset... button Thank you for your feedback!