For spark pipelines, is there a plan for dataproc serverless support? We're planning on migrating to gcp and since our workloads sporadic (only a few pipelines with intensive resources requirement, run twice monthly), using an always on dataproc cluster might not be ideal

It’s not in our immediate roadmap, but can file a GH issue to track…

Created issue at: https://github.com/dagster-io/dagster/issues/8805