Hi, need a little bit of help on docker multi cont...
# ask-community
m
Hi, need a little bit of help on docker multi container deployment I try to follow this example Deploying Dagster to Docker | Dagster but use a python module instead of python file I have 3 containers deamon, dagit and user_code -> thay all have access to a copy of dagster.yaml / workspace.yaml deamon and dagit containers are working fine but they can't reach the user_code with gRPC server. Here the dagit UI error (duplicated) :
Copy code
dagster._core.errors.DagsterUserCodeUnreachableError: Could not reach user code server. gRPC Error code: UNAVAILABLE
 File "/usr/local/lib/python3.10/site-packages/dagster/_core/workspace/context.py", line 586, in _load_location
   location = self._create_location_from_origin(origin)
 File "/usr/local/lib/python3.10/site-packages/dagster/_core/workspace/context.py", line 506, in _create_location_from_origin
   return origin.create_location()
 File "/usr/local/lib/python3.10/site-packages/dagster/_core/host_representation/origin.py", line 325, in create_location
   return GrpcServerCodeLocation(self)
 File "/usr/local/lib/python3.10/site-packages/dagster/_core/host_representation/code_location.py", line 590, in __init__
   list_repositories_response = sync_list_repositories_grpc(self.client)
 File "/usr/local/lib/python3.10/site-packages/dagster/_api/list_repositories.py", line 20, in sync_list_repositories_grpc
   api_client.list_repositories(),
 File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/client.py", line 229, in list_repositories
   res = self._query("ListRepositories", api_pb2.ListRepositoriesRequest)
 File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/client.py", line 157, in _query
   self._raise_grpc_exception(
 File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/client.py", line 140, in _raise_grpc_exception
   raise DagsterUserCodeUnreachableError(
The above exception was caused by the following exception:
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1685029509.411956807","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3260,"referenced_errors":[{"created":"@1685029509.411954997","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"
>
 File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/client.py", line 155, in _query
   return self._get_response(method, request=request_type(**kwargs), timeout=timeout)
 File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/client.py", line 130, in _get_response
   return getattr(stub, method)(request, metadata=self._metadata, timeout=timeout)
 File "/usr/local/lib/python3.10/site-packages/grpc/_channel.py", line 946, in __call__
   return _end_unary_response_blocking(state, call, False, None)
 File "/usr/local/lib/python3.10/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
   raise _InactiveRpcError(state)
user_code container log :
2023-05-25 15:38:17 +0000 - dagster - INFO - Started Dagster code server for module Dagster_proj on port 4000 in process 1
user_code container dockerfile :
Copy code
…[some pip install]
COPY app/workspace.yaml /DIS_ETL/app/workspace.yaml
COPY app_settings.yaml /DIS_ETL/app_settings.yaml
COPY app/ /DIS_ETL/app/
EXPOSE 4000
WORKDIR /DIS_ETL/app
CMD ["dagster", "api", "grpc", "-h", "0.0.0.0", "-p", "4000", "--module-name", "Dagster_proj"]
workspace.yaml :
Copy code
load_from:
 - grpc_server:
   host: user_code
   port: 4000
   location_name: "Dasgter_proj_prod"
Thanks for your time !
d
Hi Malo - does the example work for you out of the box if you make no changes?
m
nop, same issue
I'm on Ubuntu 22.04 vagrant VM
d
it sounds like there might be some networking restrictions preventing your dagit container and user_code container from communicating with each other even though they're on the same docker network. The command `dagster api grpc-health-check -h <your grpc server host> -p <your grpc serer port>' should be able to complete in your dagit pod, that might help for debugging purposes.
I just verified that the docker example works for me on my mac, so it seems likely to be something about the docker networking setup in your environment
👍 1
m
Thanks I will investigate :)
Oh, done some tests : the docker example works on my Win10 host. This is specific to ubuntu VM env. There is some content on internet but turning around certificate and secure / insecure channels.. out of my skills. I know it is'nt a good practice but as a workaround, is it possible to start automatically deamon + dagit in one container (instead of 2) ?
d
You can also run dagster dev in a single container, yeah
Dagster dev handles running dagit and the daemon for you, only one command needed
m
EDIT : found the solution
ENTRYPOINT ["dagster", "dev", "-h", "0.0.0.0", "-p", "3000"]
🎉 1