The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hey everyone,
is the following local setup possible:
Have Dagster run pyspark job using the <https://github.com/GoogleCloudPlatform/spark-on-k8s-operator|spark-on-k8s-operator from google> in a Minikube cluster on a schedule?
#mydreamsetup

Or simply running a pyspark job on minikube?

I am looking for a beginner walkthrough of some sort (spark + k8s +dagster)

Hi <@U0346U6TRFC> - I believe it should be possible - at the very least, you should be able to use `dagster-shell` to invoke it from the command line. However, I don't have familiarity with this particular setup