https://dagster.io/ logo
#ask-community
Title
# ask-community
s

Scott Hood

02/27/2023, 1:37 PM
Hey all, I created a run status sensor that looks at some of our jobs thats data relates to power bi dashboards, the code is really simple however I am starting to get errors from dagster sensor saying that the sensor tick timed out and to look at using cursors. I don't really understand why this code would cause a timeout considering its just looking to see if there is a key in a very small dictionary.... Am I missing something in my understanding of how Sensors operate that is causing sensors to run longer than anticipated?
Copy code
@run_status_sensor(
    name="power_bi_refresh_sensor",
    description="Sensor for detecting job completions and triggering power bi refresh models.",
    run_status=DagsterRunStatus.SUCCESS,
    monitored_jobs=[hive_catalog_ingest],
    request_job=power_bi_refresh_job.prod,
    default_status=DefaultSensorStatus.RUNNING
)
def power_bi_refresh_sensor(context):
    if context.dagster_run.job_name != power_bi_refresh_job.prod.name and context.dagster_run.job_name in JOB_MAPPINGS:
        run_config = JOB_MAPPINGS[context.dagster_run.job_name]
        return RunRequest(run_key=None, run_config=run_config)
    else:
        return SkipReason("Refresh skipped, job not defined.")
s

sandy

02/27/2023, 6:00 PM
Hi Scott - I definitely don't see an obvious reason that this would be causing problems. Under the covers, the run status sensor does query the database for runs. Is it possible that your database of runs is abnormally large for some reason?
s

Scott Hood

02/27/2023, 8:37 PM
Our database is running abnormally a lot at 80%... its an azure postgressdb with 4 cores.... we also enabled retention on our k8s deployment:
Copy code
retention:
      enabled: true
      schedule:
        purgeAfterDays: 90
      sensor:
        purgeAfterDays:
          failure: 30
          skipped: 7
          success: 90
3 Views