Hi, sorry if this is a naive question—when running...
# announcements
b
Hi, sorry if this is a naive question—when running
dagit
how do you specify the mode to run for a pipeline / where can I configure this? I'm getting this error running
dagit
:
Copy code
dagster.check.CheckError: Failure condition: Could not find mode default in pipeline <PIPELINE_NAME>
because I am trying to run my pipeline in a mode called "development" rather than "default" — when I run via
execute_pipeline
I'm able to pass in the mode via
RunConfig
but I can't find the equivalent way to configure in a yaml file for instance (sorry if this is in the docs and I just haven't been able to locate it). Thanks for any help! (I guess I should caveat and ask, is the only way to specify mode via the dagit UI)
a
Hi Basil. Can you show me your pipeline decorator code and how you are initializing dagit? Is this error popping up when you run dagit, or when you try to execute a pipeline in the playground?
b
Its popping up when I run
dagit
Copy code
@pipeline(
    mode_defs=[
        ModeDefinition(
            name='development',
            resource_defs={
                'postgres_db': postgres.postgres_resource
            },
            description='Mode to be used in development.',
        ),
    ],
)
I cleared my storage & history folders, and now it seems when I run
dagit
it selects the development mode automatically (no more error) so maybe it was an artifact of something in the logs?
Still curious if mode can be set via config though. Thanks!
m
if you're running in dagit, you need to use the UI to select mode
👍 1
b
Ok, good to know. Thanks!
👍 1
a
The reasoning for this is that the playground is meant to be a part of your dev process which means you really want to use the UI to toggle between modes. However, when you start using schedules to run pipelines in production or are using the python API, you need to configure and express the mode you want to execute your pipelines in which is provided by the execute_pipeline/execute_pipeline_with_presets functions
m
yep, but if it feels natural to specify mode in config, it's possible that we are organizing things wrong
b
Ok awesome thanks. The UI is actually giving me another error, not sure if it is related:
Copy code
Missing required field "resources" at document config root. Available Fields: "['exection', 'loggers', 'resources', 'solids', 'storage'].
What is the "document config root"
Also FYI my updated pipeline decorator:
Copy code
@pipeline(
    mode_defs=[
        ModeDefinition(
            name='development',
            resource_defs={
                'postgres_db': postgres.postgres_resource
            },
            description='Mode to be used in development.',
        ),
    ],
    preset_defs=[
        PresetDefinition.from_files(
            name='development',
            mode='development',
            environment_files=[
                file_relative_path(__file__, '../../utils/environments/development.yaml'),
            ],
        ),
    ],
)
And my
development.yaml
:
Copy code
resources:
  postgres:
    config:
      username: test
      password: test
      hostname: localhost
      db_name: test
Does the error mean that my yaml file is not getting loaded properly via in the presets?
(sorry to chase my question with another!)
a
yeah you need to select your presets in the UI
that should be on the right if you see the + box.
b
Ah ok—still getting used to developing with the UI I guess. Got that to work. Many thanks!!
a
glad to help! This is useful feedback for us as well so please don't hesitate to reach out if you ever have any questions with the UI
b
Will do, thanks again
s
You have discovered a real error with deleting a mode definition after creating a run with the mode. Tracking here: https://github.com/dagster-io/dagster/issues/2249
Thank you for catching and reporting!
👍 1