https://dagster.io/ logo
#dagster-plus
Title
# dagster-plus
m

Michael

04/19/2023, 3:23 PM
dagster._core.errors.DagsterUserCodeUnreachableError: dagster._core.errors.DagsterUserCodeUnreachableError: Could not reach user code server. gRPC Error code: UNKNOWN
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with
...
details = "failed to connect to all addresses"
debug_error_string = "{..."description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc"...}
I'm running a new project on Dagster cloud with a sensor that calls an external API, updates a cursor, and kicks off jobs based on the API result. It worked fine for several days, but every tick of my sensor now results in an error. You can see the full trace here. I tried redeploying my project, but the error persists.
j

jordan

04/19/2023, 4:51 PM
Hi Michael - it looks like the grpc server is exhausting CPU We’ve started seeing this a bit on sensors that use
run_request_for_partition
which is incidentally marked for deprecation in a future release - we’re still running down the exact reason but this method call has been quite CPU intensive. In the meantime, switching to invoking
RunRequest
directly might reduce the CPU enough to get this working again (and future proof you against the eventual deprecation). https://github.com/dagster-io/dagster/blob/166829d3cd80255616233771bba85b3b351a0d7f/python_modules/dagster/dagster/_core/definitions/job_definition.py#L611-L615
m

Michael

04/19/2023, 5:57 PM
That fixed it, @jordan. Thank you very much.
🙌 1
31 Views