I’m still not getting how to limit number of the the same job or uniq tags in the queue.
Now I it runs one job at the same time rest other go to the queue.
If a job get stuck the queue grows.
My goal is to have only one uniq job(tag) in the queue and skip adding more the same job(tag)
m
Marc Keeling
11/10/2021, 10:36 PM
I've been reading the documentation and haven't come across anything that sounds like it would satisfy your needs.
Maybe this could be a feature request.
p
prha
11/10/2021, 10:49 PM
We don’t have anything out of the box that does this. One way you could try to get there yourself is to implement your own run coordinator that queries the in-progress / queued runs and automatically cancels runs that have tag/job collisions in the queue upon submission.
e
Egor -
11/10/2021, 10:56 PM
Seems like I have to dig into the dagster source code 🤪
Thank you guys!
p
prha
11/11/2021, 2:09 AM
I’d start from looking at the QueuedRunCoordinator, potentially subclassing it and overriding the