Hey y all Seeing something fucky with sensors I have a senso

Question

Hey y all Seeing something fucky with sensors I have a sensor function which returns a generator decorated with ` sensor` and I have the same function copy paste passed into an explicit `SensorDefinition` call as the `evaluation fn` But the decorated function is working whereas the `SensorDefinition` instance I m constructing is not doing anything Specifically it sets up and runs as a sensor but it s not invoking the `evaluation fn` as far as I can tell Any idea what I need to do before calling `SensorDefinition` if I m not able to use ` sensor` Would be super helpful slightly smiling face

daniel · Answer

HI Michael is it possible to post the code with the SensorDefinition

Michel Rouly · Answer

Yeah give me a moment to get a minimal case One minute slightly smiling face

Michel Rouly · Answer

``` sensor name= update clustering sensor pipeline name= update clustering pipeline mode= s3 minimum interval seconds=10 def decorated sensor context SensorExecutionContext gt Generator print decorated sensor yield SkipReason No solids were added or updated def build sensor mode str gt SensorDefinition return SensorDefinition name= update clustering sensor pipeline name= update clustering pipeline mode=mode minimum interval seconds=30 evaluation fn=sensor fn def sensor fn context SensorExecutionContext gt Generator print sensor fn yield SkipReason No solids were added or updated ``` Something like this

Michel Rouly · Answer

The main reason behind this is the ability to parameterize `mode` as I ve done in `build sensor`

Michel Rouly · Answer

In this example I wouldn t ever see `sensor fn` printed out during sensor execution

daniel · Answer

just confirming `update clustering sensor fn` should be `sensor fn`

Michel Rouly · Answer

yeah typo in the example will update

daniel · Answer

and in your repository I assume you include `build sensor ` with parens where you had `decorated sensor` without parens before

Michel Rouly · Answer

Correct

prha · Answer

< Michel Rouly> are you including both functions in your repo or just one I noticed that both of the sensors have the same name and we enforce uniqueness at the repository level for sensors

Michel Rouly · Answer

I m definitely not a Python decorator expert but it looks like there s some generator unpacking going on in ` sensor` that may be relevant There s also the `decorated fn` field in `SensorDefinition` that ` sensor` parameterizes but I m not

Michel Rouly · Answer

< prha> good question only one at a time

daniel · Answer

hmmm I m seeing ticks happen when I paste in build sensor default into my list of sensors Could you post your repository too just to see if there s anything surprising there

daniel · Answer

Specifically when I turn the sensor on and go to the timeline in dagit I see a Skipped No solids were added or updated entry

daniel · Answer

I agree that unpacking should probably go inside SensorDefinition so that all sensors can take advantage of the validation but i think it should still work without it as long as your evaluation fn is a generator

Michel Rouly · Answer

Hmm We re switching our sensor code to this demo as we ve been talking and now we re not seeing any logging emitted

Michel Rouly · Answer

Give me a minute or two to double check that the demo I gave you is correct sweat

Michel Rouly · Answer

Now with our production code with ` sensor` that used to work back in our repo definition we re just actually not even seeing ticks anymore Something strange is afoot Will update

daniel · Answer

gotcha there might be some clues in your daemon output or errors on the Status page in dagit

Michel Rouly · Answer

The only logs of interest I m getting are ```2021 06 16 15 24 39 BackfillDaemon INFO No backfill jobs requested 2021 06 16 15 24 39 SchedulerDaemon INFO Not checking for any runs since no schedules have been started ```

Michel Rouly · Answer

ok fun information we blew away `$DAGSTER HOME` and restarted the daemon and dagit and our sensor is back to working

Michel Rouly · Answer

our original prod sensor decorated with ` sensor`

Michel Rouly · Answer

and after nuking `$DAGSTER HOME` a second time our `SensorDefinition` based sensor is working as expected

Michel Rouly · Answer

so tl dr invoking `SensorDefinition` directly does work but either that or simply having multiple sensors of the same names active at different times i e not concurrently may corrupt the `$DAGSTER HOME` directory

prha · Answer

I m not sure what the dagster home directory would have to do with anything to be honest What are you using for your `schedule storage` sqlite postgres

Michel Rouly · Answer

This is entirely local on a laptop defaults for everything slightly smiling face

prha · Answer

hmm I was able to switch from the decorated to the `SensorDefinition` version successfully without blowing away dagster home or even restarting the daemon

prha · Answer

as far as i can tell we identify each sensor instance by its name so there shouldn t be any artifacts stored anywhere that would cause any conflicts

prha · Answer

the daemon however takes around 60 seconds to pick up any repository changes

prha · Answer

while running

Michel Rouly · Answer

we were killing and restarting the daemon every time code was changed

Michel Rouly · Answer

we ve still got the old potentially corrupted `$DAGSTER HOME` directory if analysis on that would be helpful we backed it up before blowing it away