Hi All! Thanks for this amazing tool. I am working on a proof of concept for my team where we want to dynamically create the graph and distribute the execution over a dask cluster. Does dagster-dask currently support the experimental DynamicOutput? My pipeline fails at the moment the dynamic part should be executed, but I am not sure whether this is an error from my side or that it is just not supported yet.

The

dagster-dask

executor does not support

DynamicOutput

and I think that is unlikely to change. The

dagster-dask

executor is unique - most of our executors have a process that drives orchestration. The dagster dask executor on the other hand submits the full execution graph to dask up front and lets dask drive the orchestration. With this set-up its unclear how to modify the graph in flight dynamically.

clear, thanks!