Hi - QQ: Does Dagster has support to run the DAG’s...
# announcements
b
Hi - QQ: Does Dagster has support to run the DAG’s solids inside separated Kubernetes containers? I can see in the documentation how to run Dagster in the K8s, but I didn’t find information about spreading the solids among containers (not sure if it does make sense according to Dagster architecture). For example, do we have (or will have soon) something analog to Airflow K8s operators, or executors?
j
Currently we use dagster-celery-k8s for this. Each solid gets sent into a celery queue, from which the celery workers launch them in separate k8s jobs
n
you can check out https://docs.dagster.io/deploying/k8s_part1 for docs on how this works, including a system diagram
b
Thanks!