Hey there! I'm evaluating Dagster as a way to orch...
# ask-community
d
Hey there! I'm evaluating Dagster as a way to orchestrate some data pipelines, Airbyte, and, dbt. I come from a far simpler world were we only needed to orchestrate
dbt
and we did it via dbt Cloud. Now, we have a bunch of pipelines written in different Docker containers. We started using Argo in our k8s cluster but is becoming quite painful to use so I'd love to move some workloads to Dagster. My hunch is that I need to stat with Jobs and the
k8s
executor but wanted to check if there are any guides on how to orchestrate a bunch of container tasks. I guess, later on, we can moving towards SDAs iteratively.
c
Welcome! We have guides for deploying dagster with k8s, and we provide a k8s_job_op to directly orchestrate specific container-ed tasks. cc @daniel who might be able to provide more direct guidance
d
thanks @chris! I saw that but didn't want to jump into all the k8s stuff right away without understanding the basics
I guess I can start moving k8s jobs to Dagster k8s Job Op and then move that to Assets? Tbh, I need to look into this more as I'm still confused with all the new terminology (ops, assets, definitions, repositories, ...)