We want to work with dask dataframe as a dagaster type . What is the best strategy for running such process in dagster?

To better understand your question. Are you asking how to use dask dataframes as Dagster types? Or are you asking about how Dagster works with the dask executor? For the former, it would be like any other Dagster type, I would check out the custom type section in the tutorial. For the later, I would check out our Dagster dask integration: https://dagster.readthedocs.io/en/master/sections/deploying/dask.html

We are thinking about the former. from your reply we understand that we should follow Dagster type tutorial. Is it relevant to contribute this type to the project?

That’s right. I would use that pattern and see if it expresses everything you need of not, let us know. As far as contributing it, I’d say implement it in your pipelines first and as it becomes a recurring pattern in your pipelines file and PR and we can discuss there. Super curious to see how you all end up using it in yalls pipelines!

We have implemented most of dagster_dask_dataframe_type, hope to share the code in the future. regarding the test_date_column function in pandas test suite - what is the test checking for? the current test puts the

date

as the column name and not as value in the column