Hi everyone I m trying to work through the local Dask exampl

Question

Hi everyone I m trying to work through the local Dask example here <https docs dagster io latest deploying dask > and I m getting some errors about using an ephemeral DagsterInstance When I write the config in the Dagit playground using `filesystem` everything works great When I use the Python API and pass the YAML config like `dagster pipeline execute f dask hello world py n dask pipeline e dask hello world yaml` I get a `PIPELINE INIT FAILURE` error

Justin Swaney · Answer

More of the error message here ```dask pipeline dbd1d007 38b9 4d20 a49e bb196edf0e10 PIPELINE INIT FAILURE Pipeline failure during initialization of pipeline dask pipeline This may be due to a failure in initializing a resource or logger event specific data = error dagster core errors DagsterUnmetExecutorRequirementsError You have attempted to use an executor that uses multiple processes with an ephemeral DagsterInstance A non ephemeral instance is needed to coordinate execution between multiple processes ```

alex · Answer

alex · Answer

you just need to set `$DAGSTER HOME`

alex · Answer

we should improve that error message

alex · Answer

it works in `dagit` because we create a temporary directory based `DagsterInstance` for you if `DAGSTER HOME` is not set

alex · Answer

which certainly makes things confusing

Justin Swaney · Answer

Oh there was a mention of `$DAGSTER HOME` in the error now that you mention it ```You can configure your default instance via $DAGSTER HOME or ensure a valid one is passed when invoking the python APIs ```

Justin Swaney · Answer

I just didn t know what it meant because I didn t have to do any of that with `dagit` Thanks for the explanation

Justin Swaney · Answer

Worked like a charm tada