Also, is there any documentation on how to use Dag...
# announcements
k
Also, is there any documentation on how to use DagsterDaemonSchedulers max_catchup_runs parameter? Would be great if a default could be defined in the Helm Chart and be overriden for individual schedules... maybe its already in the code?
d
Hi Klaus - I'd be a bit surprised if you needed to use that parameter.
It's mostly there to handle the scheduler behavior if it goes down for a while and then comes back up - we put a cap on the number of schedules that the scheduler will run for a given schedule if it missed a bunch of executions
(That said, we should definitely mention it in the docs)
k
The thing is I'd have a couple of jobs that run a delta for the last 14 days every day, so even if it missed several days due to outage it should only run once in general.
d
That makes sense as a use case. One thing that is maybe relevant here is that the catch-up behavior only happens for schedules with partition sets - so if you define your schedule using the @schedule decorator (rather than daily_schedule) you wouldn’t need to worry about the catch-up parameter and it will only ever run once. This is a part of the system that we really need to document better.
k
ah ok that sounds like it would work