UPD point up configuration example that figured out just by

Question

UPD point up configuration example that figured out just by functions definition is using `hooks= slack on failure channel name my message func ` inside ` to job ` after declaring `my message func` by example from docs but unfortunately I m getting no notifications in the channel exactly on op failure =

alex · Answer

hmm did you see any mentions of the `hook` in the event log for the run you tried

yuhan · Answer

or any events like HOOK ERRORED HOOK SKIPPED or HOOK COMPLETED

Dimka Filippov · Answer

sorry for my out I m not fully sure but I will check logs twice when getting back to my office in the morning and let you know asap

Dimka Filippov · Answer

ok I m finally got my logs after every step op I have a HOOK SKIPPED log entry where it sounds like `Skipped the execution of hook hook It does not meet its triggering conditions during the execution of op name `

Dimka Filippov · Answer

do I need to provide more code for example the message function that was added to the job declaration

yuhan · Answer

what is the failure error that you expect to trigger the hook more specifically did you see HOOK SKIPPED event for the failed op as well

Dimka Filippov · Answer

ok I found out that hook does not work only when I m executing a job directly by Python with ` execute in process ` after it has been configured

Dimka Filippov · Answer

when executing with dagster binary it works I m getting notification smiley

Dimka Filippov · Answer

but it would be very nice to have notifications about fails when it is executed directly too I mean that Python stops executing only after the hook is worked out is it possible at all did anybody try to implement something similar in my Op that must fail I m requesting a downloading of a file from DataLake with a 3rd party module that is connected as a resource the Op contains try except block to check file presence before requesting it and the special I ve tried is to remove a file from source to get Op failed is it means that I forced to use in op configured notification to slack on a step of checking file presence instead of using slack on failure hook configured for the whole job only regarding executing in the process

alex · Answer

ah `execute in process` is mostly designed for testing so we default a flag `raise on error` to true so the exception bubbles all the way up to the callsite instead of the normal flow of running hooks If you set `raise on error` to `False` I think it may resolve your issue

yuhan · Answer

hook code will be invoked in process with the job execution which means if the job execution stops hooks can no longer be executed the job execution would stop when you execute in process with `raise on error=True` which is the default

yuhan · Answer

ah exactly as alex said^

yuhan · Answer

we also recommend using <https docs dagster io concepts dagit dagit|Dagit> in this case because Dagit won t stop when a job execution fails errors It instead would wrap the error and send helpful error info to the UI i e raise on error=False by default

Dimka Filippov · Answer

yes thank you I will try changing this mode and testing our team already decided to use dagster amp dagit for huge data processing jobs and to use execute in process for small jobs just because it is very fast to operate not huge data in memory

alex · Answer

other thing to note for that API is if you want to report the events amp runs to your DB you can use `instance=DagsterInstance get `

alex · Answer

defaults to an ephemeral in memory one there will be some perf overhead for writing out the events so its a trade off for recording histor y

Dimka Filippov · Answer

it s something interesting will try to read more about it first