Hello! Fairly new to Dagster and to Docker as well...
# ask-community
j
Hello! Fairly new to Dagster and to Docker as well. I'm trying to use the example from here: deploy_docker, but when I make changes to repo.py file I need to rebuild the container. There's a thread mentioning "a local bind mount" and using "QueuedRunCoordinator" or some changes to the user code docker file, but it is not very specific unfortunately. 😞 Can someone please help me understand how to setup the example project so that I don't need to rebuild image with every code change?
r
There's been previous discussion about this here: https://dagster.slack.com/archives/C01NXMEPJ7R/p1648265274203059
j
Thank you for this 🤩
c
Hi Jimmy! Fair warning that I am also not familiar with Docker, but I was able to make some changes to the Docker Example to avoid rebuilding the image with each code change. In the
docker_example_user_code
service (code here) I added a bind mount that maps my dagster source code directory to the working directory listed in the
Dockerfile_user_code
(file here). So the service snippet now looks like:
Copy code
docker_example_user_code:
    build:
      context: .
      dockerfile: ./Dockerfile_user_code
    container_name: docker_example_user_code
    image: docker_example_user_code_image
    restart: always
    environment:
      DAGSTER_POSTGRES_USER: "postgres_user"
      DAGSTER_POSTGRES_PASSWORD: "postgres_password"
      DAGSTER_POSTGRES_DB: "postgres_db"
      DAGSTER_CURRENT_IMAGE: "docker_example_user_code_image"
    volumes: # added this section here
      - type: bind
        source: /Users/claire/Desktop/deploy_docker/src
        target: /opt/dagster/app/
    networks:
      - docker_example_network
My dagster code is located in the
/Users/claire/Desktop/deploy_docker/src
directory. The Dockerfile_user_code file then has to be edited to no longer copy the code into the working directory (So commented out
COPY repo.py /opt/dagster/app
) Then, in the
dagster.yaml
file, I added an additional volume to the
run_launcher
snippet to allow the mounted code to be accessible to the run launcher:
Copy code
run_launcher:
  module: dagster_docker
  class: DockerRunLauncher
  config:
    env_vars:
      - DAGSTER_POSTGRES_USER
      - DAGSTER_POSTGRES_PASSWORD
      - DAGSTER_POSTGRES_DB
    network: docker_example_network
    container_kwargs:
      volumes: # Make docker client accessible to any launched containers as well
        - /var/run/docker.sock:/var/run/docker.sock
        - /tmp/io_manager_storage:/tmp/io_manager_storage
        - /Users/claire/Desktop/deploy_docker/src:/opt/dagster/app # This line here
After that, I was able to view dagit locally by running docker-compose up. Reloading my code via the instance status page picked up on new code changes
j
@claire Wow, thank you so much for the detailed explaining, Claire! This was exactly the what I needed! I only had to add
/c
in front of the source path since I am on Windows. And I also removed
/src
at the end of the source path. I suppose you have moved the
repo.py
file in the
src
folder?
c
yep, thats right
🌈 1