Hi Dagster Team! :dagster-yay: I am currently try...
# ask-community
s
Hi Dagster Team! dagster yay I am currently trying to mount a python file using volumes/volume mount into a user code deployment pod via helm charts in my values.yaml file I am doing this so the pod refers to python file in the volume, giving me flexibility to add more files for new user code deployments without docker build (directly uploading files into volumes and pointing the file in the pod) But I am getting this error (Looks like dagster grpc is not able to locate the file)
Copy code
dagster.core.errors.DagsterUserCodeUnreachableError: Could not reach user code server
  File "/usr/local/lib/python3.7/site-packages/dagster/core/workspace/context.py", line 555, in _load_location
    location = self._create_location_from_origin(origin)
  File "/usr/local/lib/python3.7/site-packages/dagster/core/workspace/context.py", line 481, in _create_location_from_origin
    return origin.create_location()
  File "/usr/local/lib/python3.7/site-packages/dagster/core/host_representation/origin.py", line 308, in create_location
    return GrpcServerRepositoryLocation(self)
  File "/usr/local/lib/python3.7/site-packages/dagster/core/host_representation/repository_location.py", line 523, in __init__
    list_repositories_response = sync_list_repositories_grpc(self.client)
  File "/usr/local/lib/python3.7/site-packages/dagster/api/list_repositories.py", line 19, in sync_list_repositories_grpc
    api_client.list_repositories(),
  File "/usr/local/lib/python3.7/site-packages/dagster/grpc/client.py", line 164, in list_repositories
    res = self._query("ListRepositories", api_pb2.ListRepositoriesRequest)
  File "/usr/local/lib/python3.7/site-packages/dagster/grpc/client.py", line 110, in _query
    raise DagsterUserCodeUnreachableError("Could not reach user code server") from e
The above exception was caused by the following exception:
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses" debug_error_string = "{"created":"@1657005363.580750937","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3128,"referenced_errors":[{"created":"@1657005363.580750327","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":163,"grpc_status":14}]}" >
  File "/usr/local/lib/python3.7/site-packages/dagster/grpc/client.py", line 107, in _query
    response = getattr(stub, method)(request_type(**kwargs), timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
Happy to share values.yaml file and other details on DM! thankyou
p
Hi Saurav. Are you able to connect to the pod and locate the file? Just looking for confirmation that the volume is mounted correctly…
s
Yes the file is present at the desired location, volume is mounted correctly But, upon running job dagit getting this error in run pod
Copy code
Traceback (most recent call last):
  File "/usr/local/bin/dagster", line 5, in <module>
    from dagster.cli import main
  File "/usr/local/lib/python3.7/site-packages/dagster/__init__.py", line 204, in <module>
    from dagster.core.launcher import DefaultRunLauncher
  File "/usr/local/lib/python3.7/site-packages/dagster/core/launcher/__init__.py", line 1, in <module>
    from .base import (
  File "/usr/local/lib/python3.7/site-packages/dagster/core/launcher/base.py", line 8, in <module>
    from dagster.core.workspace.workspace import IWorkspace
  File "/usr/local/lib/python3.7/site-packages/dagster/core/workspace/__init__.py", line 1, in <module>
    from .context import IWorkspaceProcessContext, WorkspaceProcessContext
  File "/usr/local/lib/python3.7/site-packages/dagster/core/workspace/context.py", line 13, in <module>
    from dagster.core.host_representation import (
  File "/usr/local/lib/python3.7/site-packages/dagster/core/host_representation/__init__.py", line 50, in <module>
    from .repository_location import (
  File "/usr/local/lib/python3.7/site-packages/dagster/core/host_representation/repository_location.py", line 9, in <module>
    from dagster.api.list_repositories import sync_list_repositories_grpc
  File "/usr/local/lib/python3.7/site-packages/dagster/api/list_repositories.py", line 6, in <module>
    from dagster.grpc.types import ListRepositoriesResponse
  File "/usr/local/lib/python3.7/site-packages/dagster/grpc/__init__.py", line 11, in <module>
    from .client import DagsterGrpcClient, ephemeral_grpc_api_client
  File "/usr/local/lib/python3.7/site-packages/dagster/grpc/client.py", line 9, in <module>
    from grpc_health.v1 import health_pb2
  File "/usr/local/lib/python3.7/site-packages/grpc_health/v1/health_pb2.py", line 38, in <module>
    create_key=_descriptor._internal_create_key),
  File "/usr/local/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 755, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: <https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates>
p
Are you running dagster version <=
0.15.0
? I think
protobuf==3.19.22
introduced a breaking change. We started including a pin in dagster
0.15.1
. You could upgrade dagster to pick up the pin or manually pin
protobuf
yourself?
s
Downgraded protobuf version and it worked! Thanks @prha