Is there a cursor-esque way to use schedules? My s...
# ask-community
Is there a cursor-esque way to use schedules? My sensor keeps timing out because the total time of the evaluation function sometimes takes longer than 60 seconds. I can't break it up because it relies on opening an SSH tunnel, which would cause a conflict with multiple sensors running at the same time
Cursor esque way meaning conditionally evaluate whether or not to actually trigger a run? Yes, I can dig up and example in a few
in that there's a cursor persisted between runs that I can easily pull and update
basically I want a "sensor" that can run longer lol
Hmm, I’m not sure how to do that. You could use asset metadata (if this is an asset job) - access the metadata from the last materialized asset in your schedule
ooh that's not a bad idea
ah so the downfall of this idea is that my assets are dynamically partitioned, so they can't be on the same job, which means they can't use the same schedule
BUT it looks like @yuhan has a PR that would resolve that
alas I think Yuhan's PR will only work for assets that have the same partitioning
@sandy I've seen some threads about setting
to higher than 60 to avoid sensor timeouts, but it doesn't appear to be taking effect
do you know where I should set that value in a helm chart for dagster cloud (hybrid deployment)?
will it work if I set it on the envvar UI?
mind asking that question is the #dagster-cloud channel? this isn't my area of expertise, but someone should be able to help you there
👍 1