Gerardo Santacruz
03/06/2024, 8:20 PMScout
03/06/2024, 8:20 PMDefinitions
object may be defined at the module scope within a single code location. If Dagster finds multiple Definitions
objects in a single Python module, it will raise an error. This is to ensure that definitions within a code location have a common namespace and must have unique names, allowing them to be grouped and organized by code location in tools. Code locations | Dagster Docs button [RFC] Workspace/repository elimination f... button runs executed from the CLI don't show up... button [dagster-msteams] make_teams_on_run_fail... button Thank you for your feedback!Gerardo Santacruz
03/06/2024, 8:23 PMScout
03/06/2024, 8:23 PMDefinitions
instances, you would need to create them in separate code locations. Each code location can contain only one Definitions
object. Here's how you can structure your project to accommodate multiple `Definitions`:
1. Separate Python Modules: Create separate Python modules (directories with an __init__.py
file) or separate Python files for each Definitions
instance. Each module or file will represent a distinct code location.
2. Top-Level Variable: In each module or file, create a top-level variable that contains the Definitions
object. For example, in my_first_file.py
, you would have:
python
# my_first_file.py
from dagster import Definitions
defs = Definitions(
assets=[...],
schedules=[...],
sensors=[...],
resources=[...],
)
And in my_second_file.py
, you would have:
python
# my_second_file.py
from dagster import Definitions
defs = Definitions(
assets=[...],
schedules=[...],
sensors=[...],
resources=[...],
)
3. Workspace Configuration: Define a workspace.yaml
file at the root of your project that specifies each code location. For example:
yaml
load_from:
- python_file: my_first_file.py
- python_file: my_second_file.py
4. Running Dagster: When running Dagster locally, you can use the dagster dev
command with the -w
argument to specify the workspace file:
shell
dagster dev -w workspace.yaml
This setup allows you to have multiple Definitions
instances, each in its own code location, while adhering to the constraint that each code location can only have one Definitions
object. Support loading Definitions from multipl... button How to use multiple definitions.py in co... button It possible to have both repositories an... button [RFC] Workspace/repository elimination f... button Thank you for your feedback!