Hey there! I'm evaluating Dagster as a way to orchestrate some data pipelines, Airbyte, and, dbt. I come from a far simpler world were we only needed to orchestrate

dbt

and we did it via dbt Cloud. Now, we have a bunch of pipelines written in different Docker containers. We started using Argo in our k8s cluster but is becoming quite painful to use so I'd love to move some workloads to Dagster. My hunch is that I need to stat with Jobs and the

k8s

executor but wanted to check if there are any guides on how to orchestrate a bunch of container tasks. I guess, later on, we can moving towards SDAs iteratively.

Welcome! We have guides for deploying dagster with k8s, and we provide a k8s_job_op to directly orchestrate specific container-ed tasks. cc @daniel who might be able to provide more direct guidance

thanks @chris! I saw that but didn't want to jump into all the k8s stuff right away without understanding the basics