19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Can one use a dagster sensor to react to an arbitr...
# ask-communityr
Rahul Dave
03/01/2023, 9:20 PMCan one use a dagster sensor to react to an arbitrary dagster event. My guess is that since asset materializations can be reacted to in sensors ( or for that matter job success or job failure can trigger a new job using a sensor), this is how dagster does it under the hood. But the docs say that user code shoukd not emit events….
Rahul Dave
03/01/2023, 9:27 PMThis would allow arbitrary processes to be spun off. My thinking is that as my machine learning models train WITHIN a dagster job( like for training different model types in sklearn in a loop where each iteration runs an op) I would like to fire off metric crunching jobs that react to an output event which is unconnected to any downstream input. I realize this could be done by doing the metric compute on a downstream op rather than a downstream job, or by using asset materialization sensors, but I am not using assets, just ops…
s
sandy
03/01/2023, 10:26 PMYou can use
context.instanceDagsterInstanceAssetMaterializationr
Rahul Dave
03/01/2023, 10:32 PMAha! So then an AssetMayerialization is a veneer over an output event? I guess since I don’t use assets I can just tail the event log for an appropriate op output event and use this in a non-standard way ( by which I mean as a not for consumption on input). Of course this means that this usecase is not a triggered sensor, but rather a sensor I could create with the original job using the python api…
Rahul Dave
03/01/2023, 10:37 PMOut of curiosity, is there a reason why creating and emitting events is not part of the public api? Is it for not wanting to support a generic event queue mechanism?
s
sandy
03/02/2023, 4:14 PMSo then an AssetMayerialization is a veneer over an output event?Step output events indicate "this op returned a value". Asset materialization events indicate "some value was placed in persistent storage".
Out of curiosity, is there a reason why creating and emitting events is not part of the public api? Is it for not wanting to support a generic event queue mechanism?Creating and emitting certain kinds of events (like AssetMaterializations and ExpectationResults) is part of the public API. For generic events, it's something we've lightly discussed but haven't developed a strong stance on.
r
Rahul Dave
03/02/2023, 7:05 PMGot it, Thanks! My understanding increases daily!!!
2 Views