https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Aaron Hoffer

06/07/2022, 5:55 PM
A couple queuing questions: • Is there a LIFO option for the queue? • Is there a way for the sensor to know how many things are in the queue? ◦ I’m running a sensor based on files in a directory but only one run can be done concurrently. I batch files into batches of up to 100 but as files come in there might be two queue runs with 50 and I’d like to run them in a single run or prevent a run to get queued until it has hit 100 or if there are no runs in the queue.
dagster bot responded by community 2
s

Stephen Bailey

06/07/2022, 6:23 PM
you can tag jobs with
dagster/priority=N
to control order of job scheduling
thankyou 1
you can definitely use a
graphql
query to list jobs that have been submitted and their current status. not sure if theres an easier way to do it than graphql.
i could see some "wait for all other jobs" logic being baked into either the sensor, or a first step in the jobs themselves. not sure if there's downsides to having a sensor hanging out and polling for other jobs to complete before requesting more runs
a

Aaron Hoffer

06/07/2022, 6:25 PM
Thanks! I’ll give the graphql query a try and if it’s not too slow that should work.
3 Views