Kris Kim
06/20/2023, 2:10 AMKris Kim
06/20/2023, 2:11 AM2023-06-20 01:51:33 +0000 - dagster.daemon.SensorDaemon - INFO - Not checking for any runs since no sensors have been started.
2023-06-20 01:51:54 +0000 - dagster.daemon.SensorDaemon - ERROR - Sensor daemon caught an error for sensor new_object_in_bucket_sensor
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dagster/_grpc/client.py", line 181, in _streaming_query
method, request=request_type(**kwargs), timeout=timeout
File "/usr/local/lib/python3.7/site-packages/dagster/_grpc/client.py", line 169, in _get_streaming_response
yield from getattr(stub, method)(request, metadata=self._metadata, timeout=timeout)
File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 426, in __next__
return self._next()
File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.DEADLINE_EXCEEDED
details = "Deadline Exceeded"
debug_error_string = "{"created":"@1687225913.860669353","description":"Deadline Exceeded","file":"src/core/ext/filters/deadline/deadline_filter.cc","file_line":81,"grpc_status":4}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dagster/_daemon/sensor.py", line 541, in _process_tick_generator
sensor_debug_crash_flags,
File "/usr/local/lib/python3.7/site-packages/dagster/_daemon/sensor.py", line 671, in _evaluate_sensor
instigator_data.cursor if instigator_data else None,
File "/usr/local/lib/python3.7/site-packages/dagster/_core/host_representation/code_location.py", line 852, in get_external_sensor_execution_data
cursor,
File "/usr/local/lib/python3.7/site-packages/dagster/_api/snapshot_sensor.py", line 72, in sync_get_external_sensor_execution_data_grpc
timeout=timeout,
File "/usr/local/lib/python3.7/site-packages/dagster/_grpc/client.py", line 397, in external_sensor_execution
custom_timeout_message=custom_timeout_message,
File "/usr/local/lib/python3.7/site-packages/dagster/_grpc/client.py", line 185, in _streaming_query
e, timeout=timeout, custom_timeout_message=custom_timeout_message
File "/usr/local/lib/python3.7/site-packages/dagster/_grpc/client.py", line 138, in _raise_grpc_exception
) from e
dagster._core.errors.DagsterUserCodeUnreachableError: The sensor tick timed out due to taking longer than 60 seconds to execute the sensor function. One way to avoid this error is to break up the sensor work into chunks, using cursors to let subsequent sensor calls pick up where the previous call left off.
2023-06-20 01:52:38 +0000 - dagster.daemon.SensorDaemon - WARNING - Sensor new_object_in_bucket_sensor was started from a location pm_dagster_project that can no longer be found in the workspace. You can turn off this sensor in the Dagit UI from the Status tab.
Kris Kim
06/20/2023, 2:12 AMjohann
06/21/2023, 8:34 PM