Juan Freire
06/27/2023, 2:21 PMgrpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.DEADLINE_EXCEEDED
details = "Deadline Exceeded"
debug_error_string = "{"created":"@1687870896.003832471","description":"Deadline Exceeded","file":"src/core/ext/filters/deadline/deadline_filter.cc","file_line":81,"grpc_status":4}"
Checking on the code server deployment, we can see that it consumes a big amount of CPU, in contrast to other code servers under the same dagster instance. However, I cannot reproduce this problem locally, as the sensor will run with no problems, and neither can I see any logs from the sensor using the "experimental schedule/sensor logging view" feature.
I´m running on dagster 1.3.3
sean
06/27/2023, 2:55 PMJuan Freire
06/27/2023, 4:13 PMcontext.log
in the sensor UI.
Also, I´ve tested running a script similar to the sensor evaluation directly on the code server's pod, which I belive (please correct me if I am wrong) is where the sensor evaluation runs. The script runs normally with the same access credentials.Juan Freire
06/27/2023, 7:40 PMJuan Freire
06/27/2023, 7:41 PMsean
06/27/2023, 7:46 PMJuan Freire
06/27/2023, 8:20 PM