The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hello,
Our sensors are consuming more and more CPU over time until the entire machine is stuck.
We are using dagster 1.2.6 and deploying via k8s.

We have reviewed our code and cant find any reason for this “CPU leak”. Is this a known issue with the sensors? are we misusing them somehow?


Sensor definition:
```production = SensorDefinition(
    name='sensor',
    evaluation_fn=SensorClass().run_sensor,
    job=prod_job,
    minimum_interval_seconds=60 * 10,
)```
:pray: thanks for any help

Hmm, not that I know of. What's the general logic of your evaluation function?

1. query a GCS bucket to list all config files
2. iterate over config files that are newer then the cursor file:
    a. download file and load it into a class
    b. query its relevant data from postgres
    c. create run requests
3. create skip reason if no run was found 

But anyway what could we have possible done that the same process is capping its CPU over time.
It seems to me as if the GRPC process i supposed to call the sensor and clear its resources when done, whether its memory that was used or its CPU.