Hi guys, I'm facing an issue when i'm using dask executor on my job then filling the dask config as follow :

execution:
  dask:
    config:
      cluster:
        pbs:
          queue: mpi_4
          project: dagster
          walltime: '00:05:00'
          memory: '120gb'
          cores: 28
          resource_spec: 'select=4:ncpus=28:mem=120gb'
          n_workers: 1

I got this error :

raise DagsterInvalidConfigError(
dagster.core.errors.DagsterInvalidConfigError: Error in config for job
    Error 1: Received unexpected config entry "dask" at path root:execution. Expected: "{ config: { cluster: { existing: { address: (String | { env: String }) } kube(/home/datawork-semaphore-exp/conda/conda_envs/semaphore-scripts-1.0.11) semexp@datarmor3:/home1/datawork/semexp/workspace/dagit/semaphore-dagster-config/pipelines>     Error 2: Missing required config entry "config" at path root:execution. Sample config for missing entry: {'config': {'cluster': '<selector>'}}

And my job looks like that :

@job(
    name='template_runner_job',
    resource_defs={"io_manager": fs_io_manager},
    executor_def=dask_executor
)
def template_runner_job():
    template_runner_op()

Hi Thomas. I think you can drop the

dask

in your config:

execution:
  config:
    cluster:
      existing:
        ...

Are you using the Launchpad in Dagit to compose the config? If not, you may find the autocomplete / config validation feature useful.

If i'm using the launchpad to compose the config i can't insert

dask

because of a control on the interface that doesn't allow me this keyword

cc @chris Thomas, can you share the error that you’re getting?

Pretty sure using dask executor with config is broken right now on jobs - https://github.com/dagster-io/dagster/pull/5848 will fix

@chris Hi, I pick the change you've made in

<https://github.com/dagster-io/dagster/pull/5848/files/8efc6be1cc2e6795e56fb002846b9d17ef694720#diff-f4477f5a8ae52e60950dc28d9f884f39f9912e850e02692dce9b0fb325ae280b|dagster_dask/executor.py>

After you've picked the change in that PR - you should not have

dask

in your config. Since jobs only have one executor, there is no need to additionally specify the executor name in execution config. Since

executor_name

is no longer part of the schema, you're getting an error. If you get rid of the

dask

keyword, it should work