run_key, sensors and idempotence. Hi, is there a notion of ‘successful execution’ that can be leveraged inside sensors? I am interested in the case where a sensor triggers the relevant pipeline but because - say - some other artefact is not yet available (or some other condition is not met), then the run is marked as ‘void’. The sensor is then expected to trigger again later. How can this be done? Thanks

Hi Marco - the way I would think about implementing this: • Include a solid in your pipeline that either fails or doesn't execute at all for "void" runs • In your sensor, use methods on

context.instance

to find whether the that step failed or ran in the relevant pipeline runs. We could do a better job documenting and exposing the methods on the instance, but here they are: https://docs.dagster.io/_apidocs/internals#dagster.DagsterInstance You may need to use semi-private APIs. I believe you can do something like

step_stats = context.instance.get_run_step_stats(run_id, step_key)
status = step_stats.status

if status.value == "SKIPPED": # or "FAILED"
    # do something

thanks Sandy, that’s helpful. As a follow up question: how does the status ‘SKIPPED’ or ‘FAILED’ come about? In other words how do I ‘fail’ the solid?

You can do:

from dagster import Failure

@solid
def my_solid(_):
   yield Failure()

Got it, thanks!

Ah - that's right. You'd have to set

run_key=None

or use a unique run key if you wanted to be able to do two different submissions

Thanks. What’s the best way to debug a sensor, by the way?