hello dagster community I have one quick question: I have job + scheduler to run this job every minute; is here some way to check is this job running now and do not start new one if it is (so just skip next schedule run)? thank you in advance)

Hi @Dmytro Tsyliuryk did you mean a programmatic way to tell if a job is running?

im not sure about the way to do that, but as equipment from Airflow its max_active_runs, so you can put number of jobs that can run in parallel

just to clarify, Airflow’s max_active_runs applies to the number of the same job or the number of concurrent runs for all jobs?

just to clarify, Airflow’s max_active_runs applies to the number of the same job or the number of concurrent runs for all jobs?

the same job

Hi @Dmytro Tsyliuryk sorry for the delayed response (I was out later last week). • The multiprocess_executor config max_concurrent is used for setting concurrent running ops inside one job run. • We also have a way to set max concurrent (across all jobs) in run coordinator: https://docs.dagster.io/deployment/run-coordinator#run-limits Unfortunately, what you are looking for (setting max concurrent runs for the same job) isn’t supported out-of-box at the moment. I’d suggest to write a custom logic in your schedule, something like:

@schedule(...)
def my_schedule(context):
	run_records = context.instance.get_run_records(
	    PipelineRunsFilter(
	    	pipeline_name="your_job_name", 
		    statuses=[PipelineRunStatus.STARTED],
		)
	)
	if len(run_records) == 0:
		yield RunRequest(...)

cool, i see now. thank you, that should work )

@Dagster Bot discussion How to not schedule a new run if a same job run is running

Question in the thread has been surfaced to GitHub Discussions for future discoverability: https://github.com/dagster-io/dagster/discussions/8414