https://dagster.io/ logo
Join Slack
Powered by
Hi all! I am doing a multi-container Docker deploy...
# ask-community
n
Hi all! I am doing a multi-container Docker deployment. But I am having issues configuring my Dagster instance to use the dagster-daemon process running in a seperate container. Similar to this example. How do you have to configure your dagster.yml in that case ? The dagster container and the dagster-daemon container are on the same network and accessible. Thanks for your help!
j
Hi @Nicolas Nguyen does the dagster.yaml in the example not work for you? what kinds of errors are you seeing? https://github.com/dagster-io/dagster/blob/0.15.6/examples/deploy_docker/dagster.yaml
n
Hey Jamie, Thank you for your response! Please correct me if I'm wrong, the only relevant part in that config for the dagster-daemon is the run_launcher, right ? (for the rest I just want to keep the default for now)
What does the key module: dagster_docker do in that example ?
Also, my dagster-daemon process runs on host docker-dagster-daemon, I don't understand how to integrate that in the dagster.yml file.
j
module: dagster_docker
tells dagster to use the docker run launcher. if you remove everything from the example dagster.yaml except for the run_launcher section, what kind of behavior do you see?
n
With 
module: dagster_docker
only, I get the following error:
With this dagster config : 
run_launcher:
module: dagster_docker
class: DockerRunLauncher
config:
network: my-network
container_kwargs:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /tmp/io_manager_storage:/tmp/io_manager_storage
Both containers (dagster and dagster-daemon) are running and accessible on the network my-network
d
Hey Nicolas - there's a service in that example that runs a gRPC server with dagster code in it, which also provides an image to use using the DAGSTER_CURRENT_IMAGE env var: https://github.com/dagster-io/dagster/blob/0.15.6/examples/deploy_docker/docker-compose.yml#L21-L35
"By setting DAGSTER_CURRENT_IMAGE to its own image, we tell the run launcher to use this same image when launching runs in a new container as well." - from the error message there, it looks like you may be missing that part
n
That was actually the issue thanks a lot Daniel!
condagster 1
Open in Slack
PreviousNext
Powered by