Hi, In the <Executing on Dask documentation> you ...
# announcements
s
Hi, In the Executing on Dask documentation you state that -
you may find that invoking Dask or Pyspark directly from within the body of a solid’s compute function is a better fit than the engine layer covered in this documentation.
Do you have an example to invoke Dask execution compute function from within a
solid
, we are running on a
cloudera
environment and need to run the solid process with dask-yarn. any advice would be appreciated.
n
Hi @sephi - I just went back through and I don’t think we currently support this because we directly pass the dask resource configuration to the dask client here: https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-dask/dagster_dask/engine.py#L83 - I filed https://github.com/dagster-io/dagster/issues/2273 for us to take a look at this