Hi folks, I'm looking for a way to have the docker launcher start runs from the code container itself not the daemon, since I'd like to spread the load on several different nodes in our swarm. The docs on the subject suggest that this is possible, any clues on how to go about this?

We don’t currently have a feature that does this - apologies if the docs are misleading. The default run launcher lets you make launch runs on the code container from the daemon, but I’m not aware of a way to launch runs from a different container besides the daemon.

so I can have multiple code containers on different nodes, and the daemon can launch runs on those code containers, i suppose in separate processes, but not spin up a separate container, is that correct?

Would it be possible to write a

DockerSwarmRunLauncher

? And how complicated of a process is that?

Jesse I don’t think that would quite work out of the box right now to have multiple code containers at once on different nodes - but sounds possible with some changes

What changes are we talking about, is that something we can do on our end, or it's a feature that needs to be implemented in the framework?

Danny a DockerSwarmRunLauncher sounds like not a huge lift to me as long as it has an api for launching containers, you could take a look at the DockerRunLauncher as a reference, it’s not huge - https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-docker/dagster_docker/docker_run_launcher.py#L99

so if we want to scale horizontally we'd need to either build a custom DockerSwarmRunLauncher or another custom run launcher that spins up runs from distributed code containers?

Or use k8s or ecs which are better equipped to scale horizontally out of the box, but I imagine you were already aware of those options

yes, our infra runs on docker swarm, so that will be very inconvenient

You could file a feature request issue for the docker swarm run launcher if writing one is daunting - there may be others out there who would be interested in such a thing and we could incorporate it into future prioritization

alright, thanks Daniel