https://dagster.io/ logo
Title
s

stefan hansan

11/18/2022, 8:05 PM
Hey, quick question. Can ops be run from sensors? According to the documentation, "Ops are used within a job or graph." So right now I have a sensor that runs a slight variation of the code contained in an op, and they produce very similar results. It would be great if the sensor could just call that op directly instead of having to do run the code within the sensor itself. Is there a good way to invoke it from a sensor? Thank you!
c

claire

11/18/2022, 8:09 PM
Hi Stefan. Unfortunately not, ops can only be run within jobs/graphs. This is because they generate events, and events must be exist within the scope of a run. You could instead make the sensor yield a run request that will invoke the op.
s

stefan hansan

11/18/2022, 8:12 PM
hi claire, thanks for the quick reply! That makes sense! So right now, that sensor is calling op B that is using the data calculated within the sensor. If i wanted to call an op (op A) that calculates the data from the run request, could i then feed it into op B from that same run request? and what would the syntax look like? I often get confused on how the run_config would look. Like does run_config= {ops:{'opA'}, {'opB'} mean B is getting passed the data from A? Thanks!
c

claire

11/18/2022, 8:58 PM
Hm... in order for the suggestion I mentioned to work, you would have to make op B accept an input from op A. The run config you shared would pass a different run config object to op A and op B each. Instead, to make op B accept data from op A, you would do something like this:
@op(config_schema={...})
def op_a():
    return ...

@op
def op_b():
    return ...

@job
def my_job():
    op_b(op_a())
:dagster: 1
s

stefan hansan

11/18/2022, 9:07 PM
Sorry, just to clarify, that job would be launched from the sensor's run request then? I currently do employ this strategy of passing data from one op to another through a job, but im not sure of how to do that from a sensor. I apologize, its hard to articulate what i mean 🥴
c

claire

11/18/2022, 9:08 PM
Yes, that's right, the sensor would yield a run request that kicks off this job
:rainbow-daggy: 1
s

stefan hansan

11/21/2022, 1:42 PM
That makes sense, Thank you!