https://dagster.io/ logo
Title
d

Danny Jackowitz

03/04/2022, 8:44 PM
👋 Question regarding using cursors with sensors as described here: https://docs.dagster.io/concepts/partitions-schedules-sensors/sensors#sensor-optimizations-using-cursors What are the transactional guarantees around
update_cursor()
and yielded run requests? Is there anything we need to watch out for that could result in missed runs while using cursors (e.g. the cursor is updated but the run never actually happens)? Or is it guaranteed that the cursor update will never be recorded without the run request also?
d

daniel

03/04/2022, 8:47 PM
Hey Danny - the cursor isn't updated until after the end of the tick that creates the runs (so calling update_cursor doesn't immediately update the cursor right there, it's passed back through to the daemon and written once its gone ahead and launched the runs)
d

Danny Jackowitz

03/04/2022, 8:49 PM
Perfect. I figured that was the case but didn’t see anything that explicitly defined the semantics and didn’t want to assume and silently drop data. Thanks, @daniel.
d

daniel

03/04/2022, 8:50 PM
we should make that clearer, yeah. @Dagster Bot docs clarify guarantees around sensor cursors and failure recovery
🙇 1
d

Dagster Bot

03/04/2022, 8:50 PM