With the

dask_executor

, how do I connect to an existing Dask cluster? The documentation (https://docs.dagster.io/_apidocs/libraries/dagster_dask#dagster_dask.dask_executor) seems to suggest that I can configure

address

on the

local

type. But, doing so results in a

TypeError: __init__() got an unexpected keyword argument 'address'

. Here's how I currently have the config.

execution:
  dask:
    config:
      cluster:
        local:
          address: <tcp://xxxxxxxxxxxxx:8786>

When using

local

,

dask_executer

is executing a

LocalCluster

so you need to use LocalCluster parameters.

So, that means it's not possible to connect to an existing cluster, then?

Before that

dask_executer

runs only with

local

mode, so it was initialized directly with

Client

and has the parameter address. Now,

dask_executer

can run with serval clusters (e.g. yarn, kubs...). so the

local

cluster is no longer executing directly with

Client

, only with

LocalCluster

.

Hmm, looking at the

dagster_dask

code, I see that cluster objects like

LocalCluster

are being created and then passed to

Client

. The problem with that though is there's no way to connect to an existing scheduler.