Hi, What is the best way to debug a dagster gRPC server that I've kicked using the API, following this example https://github.com/dagster-io/dagster/tree/0.11.0/examples/deploy_docker ? The gRPC server container is not logging anything as far as I can tell, while

dagster api grpc-health-check

returns nothing. The error I am getting in dagit is:

grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses"
	debug_error_string = "{"created":"@1616415994.552532757","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":5396,"referenced_errors":[{"created":"@1616415994.552531220","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":397,"grpc_status":14}]}"

Thanks for your help

Hi Lewis - typically if the gRPC server runs into an error while starting up there should be some kind of error. It’s also possible that the configuration in your workspace.yaml isn’t pointing at the right host or port - if you post your workspace file we could take a look. Do you run into this error when you run the example out of the box?

Hi Daniel, here is my workspace.yaml

load_from:
  - grpc_server:
      host: localhost
      port: 4000
      location_name: "pipelines"

I've taken parts from the example and applied them to my repo

Yeah, typically if there's an error in the gRPC server it should be there in the container logs. Is your docker-compose file pretty similar to the example?

Yeah it's pretty similar to the example

Do you get the same error if you go to reload the repository locaiton in dagit? On the status page there's a reload link

That was it, thanks so much for your help!

np! I think we should make the gRPC server not expose port 4000 to localhost in the example, that's confusing since it's not what dagit uses. Will make that change!

Is there any documentation on how grpc_sever host is resolved from workspace.yaml?

that hostname is set by docker-compose (it's the name of the docker-compose service). Is your AWS setup also using docker-compose?

Created issue at: https://github.com/dagster-io/dagster/issues/3919

No it wouldn't be using docker-compose. I'm effectively managing that myself, launching each container separately. At the moment I'm thinking of hosting the gRPC server, exposing it at say https://my-domain/dagster-grcp . Would I then be able to specify a workspace.yaml of :

load_from:
  - grpc_server:
      host: <https://my-domain/dagster-grcp>
      location_name: "pipelines"

?

I'd expect that work although you shouldn't need the 'https://'

On that basis, do you know why it doesn't work when I provide this workspace.yaml:

load_from:
  - grpc_server:
      host: localhost
      port: 4000
      location_name: "pipelines"

I'm just hoping to validate that it'll work locally before I move the build into AWS. Is there any code/docs that you can point me to about the dagster grpc_server? I just can't find anything online to inform me about what's going on behind the scenes Thanks again for all your help on this

np! I think it's because your dagit doesn't have access to localhost since it's running in docker / localhost doesn't resolve to the same place within that container as it does in your browser

Oh okay, that makes sense. I tried the config from here https://dagster.slack.com/archives/CCCR6P2UR/p1616425831143900?thread_ts=1616416315.128200&cid=CCCR6P2UR and got the following error:

dagster.check.CheckError: Invariant failed. Description: must supply either a socket or a port

. Not sure how I'd get around that or if supplying 80/443 for http/https would work

Yeah, it needs you to specify the port on which the server is listening - that can be 80 if your server is set up to be listening on that port.

Thanks Daniel, that's helped a lot