My understanding is that this limits the number of concurrent runs, not the number of queued runs, right?
d
daniel
06/21/2021, 2:18 PM
Oh, maybe I don't understand the use case. What are you hoping will happen to runs that are submitted when there are already that many queued runs, a failure?
b
Ben Torvaney
06/21/2021, 4:55 PM
Probably a cancellation? My issue is that some jobs are idempotent, so repeating them in unnecessary. It’d be nice to be able to express this in dagster so that certain jobs are queued at most one time.
That was my thinking, at least. It sounds like defining a custom coordinator would be the best course of action here?
d
daniel
06/21/2021, 4:57 PM
Yeah, a custom run coordinator would probably be the way to go there