Hi All, Is there a way to identify a adhoc run of a job, which runs once a day based on a cron schedule?

It should have the

solid_selection: *

tag

Hi @Kalyan katamreddi have you checked out our schedules page yet? https://docs.dagster.io/concepts/partitions-schedules-sensors/schedules schedules are built so that you can run a specific job on whatever time cadence you want

Hi @jamie: Thanks for the response. I am already using schedules. My use case: I have a job that runs once every day using schedules. But if a adhoc run(manual run) instead of a schedule run happened, is there a way to capture those runs?

If you have a job called

job_a

on a schedule that runs once a day, but then you manually do a run of

job_a

you should be able to see every run on

job_a

(ie all runs triggered from schedules and the manual run) in the Runs tab for the job in dagit (click on the job in the left sidebar and go to the Runs tab). Or if you go to the Status page (top nav on the right) you should see a list of every job and an overview of the status of the past few runs. you can click into the job from there are also get to the Runs tab. Let me know if there's other behavior you'd like to see and I can help out!

I am expecting to capture this behavior from a sensor, using something like @run_failure_sensor or @run_status_sensor and send an email to the user. is this possible?

yeah! in it's most basic form a

run_failure_sensor

will run for every job execution that fails (regardless of the job name and how it was triggered). You can narrow it down so that the

run_failure_sensor

only triggers for a specific job. Using the example above, if you set up a

run_failure_sensor

to monitor

job_a

the sensor would run whenever

job_a

fails (regardless of if the run was started by a schedule or by someone manually running it)

Yes!! I am already leveraging these sensors, and these sensors are working just as expected. But i just wanted to differentiate between manual run and a schedule run

oh i see, so for example, you want a sensor that detects when

job_a

fails from a scheduled run and another sensor that detects when

job_a

fails from a manual run?

exactly, thats my requirement

ok cool thanks for all the explanation while i got caught up! I have a couple ideas for you: in your failure sensor, the argument you get is a context object (api docs)

@run_failure_sensor
def my_failure_sensor(context):
   pass

there is a lot of information within the context that might be useful to you. I'm not sure if there is a specific attribute in the context that tells you who/what executed the job, but if there is, then you should be able to just look at that attribute and execute accordingly If there isn't an attribute that tells you what you need to know, here's my alternative solution: In the schedule that executes

job_a

you can attach a specific tag to the run

# pseudocode 

@schedule(job_a, cron_schedule="...")
def job_1_schedule():
   return RunRequest(
      run_key=None,
      run_config={},
      tags={"scheduled": "true"}
   )

the important thing here is the

tags

argument. This tag will only be set for the scheduled executions of

job_a

(unless someone manually running

job_a

adds this tag in the dagit launchpad). Then in your failure sensor, you can get information about the run in from the context and find the tags that are on the run. My guess is it would be under

context.pipeline_run

but i'm not 100% sure about that

Thanks for the solutions, will try it out

hmmmm i'm not sure, that's not something i've tried before. For more info: the

context

object that an op receives is created at execution time by the dagster internals, so maybe the run tags are included in that. You could root around a bit in the context object (pdb is one of my favorite tools for this) and see if you can find them. Given our general approach to tags I wouldn't be surprised if run tags are not accessible in an op though