https://dagster.io/ logo
Title
e

Egor -

11/10/2021, 7:19 PM
Hey Team, There is a way to limit only one job or tag in the Queue?
m

Marc Keeling

11/10/2021, 7:35 PM
Not sure this is what you are going for but this might help https://docs.dagster.io/deployment/run-coordinator#limiting-run-concurrency
e

Egor -

11/10/2021, 7:58 PM
I’m still not getting how to limit number of the the same job or uniq tags in the queue. Now I it runs one job at the same time rest other go to the queue. If a job get stuck the queue grows. My goal is to have only one uniq job(tag) in the queue and skip adding more the same job(tag)
m

Marc Keeling

11/10/2021, 10:36 PM
I've been reading the documentation and haven't come across anything that sounds like it would satisfy your needs. Maybe this could be a feature request.
p

prha

11/10/2021, 10:49 PM
We don’t have anything out of the box that does this. One way you could try to get there yourself is to implement your own run coordinator that queries the in-progress / queued runs and automatically cancels runs that have tag/job collisions in the queue upon submission.
e

Egor -

11/10/2021, 10:56 PM
Seems like I have to dig into the dagster source code 🤪 Thank you guys!
p

prha

11/11/2021, 2:09 AM
I’d start from looking at the QueuedRunCoordinator, potentially subclassing it and overriding the
submit_run
method
👍🏼 1
e

Egor -

11/11/2021, 2:35 AM
Awesome! Thank you man!