https://dagster.io/ logo
Title
d

David Gasquez

01/21/2023, 5:58 PM
Heya! Is there a way I could easily run a docker container as an op locally? I've tried with the docker executor but I'm struggling to find examples. I might be confusing terms (executors vs ops vs jobs ...) too. I'm trying to port some Argo tasks (docker images) to Dagster and want to try things locally.
d

daniel

01/23/2023, 3:17 PM
Hi David - to confirrm this is a Docker container that isn't running Dagster Python code? We're just in the process of adding a docker_container_op (similar to k8s_job_op: https://docs.dagster.io/_apidocs/libraries/dagster-k8s#dagster_k8s.k8s_job_op) - that will probably be ready in the release next week due to an offsite happening this week
d

David Gasquez

01/23/2023, 5:31 PM
awesome! that's exactly what I was looking for 🙂
as I'm coming from Argo. I was planing on using the k8s op on prod and the Docker one on dev. Does that makes sense?
d

daniel

01/23/2023, 6:13 PM
that could work, yeah - although both of those ops lose some of the benefits of dagster since you don't really end up taking advantage of the programming model
d

David Gasquez

01/23/2023, 6:44 PM
I know 😕 The thing is we have a bunch of potential ops already Dockerized and working
My plan was to try Dagster as an Argo replacement and then iterativelly move towards SDAs in the places we can
👍 1
d

daniel

01/23/2023, 6:44 PM
you could even get fancier and write your own op that selectively calls execute_docker_container vs. execute_k8s_job depending on configuration on the op
d

David Gasquez

01/23/2023, 6:45 PM
the main "issue" is that we have a bunch of non Python tasks that can't work on Dagster
d

daniel

01/23/2023, 6:45 PM
that makes sense and is a reasonable use case for those ops
d

David Gasquez

01/23/2023, 6:45 PM
sweet! I'll test the new Docker op when it comes out
tbh I'm still wrapping my head around all the Dagster concepts
I come from simple Dockerfiles cronjobed in Argo 🙈
Do you know anyone or a project that went through this migration?
d

daniel

01/23/2023, 6:47 PM
There are a fair amount of users using k8s_job_op for this purpose, yeah - it's the main reason we added it
d

David Gasquez

01/23/2023, 6:47 PM
awesome to know! wouldn't like ot be the only weird one using that approach
alright, thanks a lot @daniel! this is super helpful 🙂 I'm excited for the next release. Enjoy the offsite
:condagster: 1