Is there an env variable or config I can set to prevent timing out waiting for assets to load for more than 3 minutes?

/Users/myuser/mycompany/sample-dagster-flow/.venv/lib/python3.9/site-packages/dagster/_core/workspace/context.py:548: UserWarning: Error loading repository location sample:Exception: Timed out waiting for gRPC server to start after 180s with arguments

I have already set

DAGSTER_GRPC_TIMEOUT_SECONDS

to 360 but it did not work

there is! https://docs.dagster.io/deployment/dagster-instance#grpc-servers

where do i set this?

This would go in a

dagster.yaml

file in a folder that the DAGSTER_HOME environment variable is set to

I don’t have that environment variable set as far as I can tell

Got it - we're actually working on some improvements that will make it so that you don't have to mess around with environment variables for this, but for now you'll want to set that env var to some folder and put a dagster.yaml in it with that config

gotcha okay - yeah I really already think it’s going to be a big lift to get my internal engineers to learn a whole new framework (dagster) for their jobs, so having extra files and configs to worry about for local configuration doesn’t help right now.

We have immediate plans to make the environment variable and folder part of those steps go away, but the plan would still be to configure things using a dagster.yaml file in the folder where you run

dagit

- is there another form of configuration that you'd prefer?

in regards to another form of configuration, I’m not sure - still in the early stages of the POC (just trying to get something to work) so I don’t want to say too much before I’m a little more experienced, but it has been surprisingly difficult to get up and running relative to other orchestration systems, between figuring out ops, jobs, resources, then an additional config to load the dagster resources (the various

__init__.py

files. As far as getting it below 3 minutes - that’s how long the dbt cloud job takes, so not sure what we can do there.

Makes sense - sounds like kind of a thousands of papercuts situation. I'll re-post here as soon as that dev flow i mentioned that doesn't require any env vars is ready (it'll be a

dagster dev

command that just looks for a dagster.yaml file in the folder where you ran it from and spins up dagit using it)

yeah 100% that’s what it is — thanks!

Hey @Sam Werbalowsky - just letting you know that with 1.1.11 that just shipped we included that

dagster dev

command - you can now set that config we talked about earlier in a dagster.yaml file in the same folder that you run the command from, more info here: https://docs.dagster.io/deployment/guides/running-locally

awesome, will take a look - thank you!

Getting a similar error in dagster cloud using kuberneters. Where would I update timeout setting for cloud?

there's a

serverProcessStartupTimeout

property here that controls that: https://artifacthub.io/packages/helm/dagster-cloud/dagster-cloud-agent?modal=values-schema&path=workspace.serverProcessStartupTimeout - before increasing that though I would suggest verifying that it's actually expected for your code to take more than 3 minutes just to load your definitions, as in my experience that can happen if there is a very large asset graph (or many asset graphs) being loaded but is somewhat unusual

Hey Daniel, we are using Dagster Cloud Hybrid. Do I still need to edit the values.yaml file or there's some other way to do it via UI?

That works the same as in open source - they both have a helm chart

Okay. I see the values.yaml doesn't have this field so I will add

serverProcessStartupTimeout

. I only see 2 values,

enabled

and

servers

. Do I need to toggle

enabled

to true as well?

that looks like a screenshot from the open source helm chart which wouldn't apply here - information on the cloud helm chart is here: https://docs.dagster.io/dagster-cloud/deployment/agents/kubernetes/configuration-reference#viewing-the-helm-chart

Thanks! Still learning my way around k8s and helm charts.. I ran this command finally..

helm upgrade \
   --install user-cloud dagster-cloud/dagster-cloud-agent \
   --namespace dagster-cloud \
   --set workspace.serverProcessStartupTimeout=1200

I got a response that my release has been updated. But when I try to see the updated values in helm chart by running the

helm show values

command, the

workspace.serverProcessStartupTimeout

is still empty which makes me think that I did something wrong.

sounds like you might want the 'helm get values' command: https://helm.sh/docs/helm/helm_get_values/#helm