Is there a way to return a `RunRequest` that does ...
# ask-community
l
Is there a way to return a
RunRequest
that does nothing? It seems that a sensor's cursor is advanced only if a RunRequest is returned. However, we have a few use cases where we'd like to run a function (slack notification) in response to a multi-asset sensor, but have no associated jobs or assets to run.
D 1
z
Does
context.update_cursor
not work if you don't yield a
RunRequest
? I was under the impression that it does, regardless of if you yield a SkipReason or a RunRequest
I also would like to start leveraging sensors for lightweight function execution without having to request a whole run or go through the hoops of writing a custom RunLauncher to execute some code locally instead of spinning up a whole ECS task
j
been a while since i’ve been deep in sensor code, but i think
context.update_cursor
should do the trick
l
hmm i've been calling
advance_all_cursors
, is that different from
context.update_cursor
?
j
i think that should be the same taking a step back - what are you trying to do with the sensor? would a run status sensor do the trick so that you dont need to manually modify the context yourself?
l
the goal is to provide an interface for users to subscribe to events within the data pipeline, where an event is a combination of several assets being materialized at least once. I've implemented this using the
multi_asset_sensor
decorator to create a sensor that listens on the configured set of assets. Each sensor tick, if all of the assets have materialized at least once since the last time the sensor fired, it will execute a a user-provided function (with the expectation that the function is lightweight, disposable, stateless, etc... primarily will be notification-related), plus optionally returning a
RunRequest
to initiate runs of the configured jobs or request assets for that multi-asset sensor. It actually works fine if someone configures a job or request asset, but I noticed that when neither is provided, the user-provided function executes every tick if all of the monitored assets have materialized at least once, and none of the monitored assets have materialized since the last time that all of them were materialized.
j
ah interesting. Does returning a
SkipReason
in that case not work?
l
ah, it does! so semantically a SkipReason is sort of a "do nothing but commit cursor state" signal...
j
yeah i think so. probably good to do a double check on cursor state - but i’m like 99% sure that’s what it does
i can dig into the source code and find the real answer if you want
l
Just checked, the cursor does get updated, i think the main feedback is that "Skip" is not really the behavior more like... "Pass"?
j
yeah, i suppose the idea is like “skip launching any runs”
🌈 1