https://dagster.io/ logo
j

Josh Karlin

01/08/2021, 6:01 PM
hi, is there a success/failure hook for a whole pipeline? i can only see them at the solid level
1
a

alex

01/08/2021, 6:18 PM
cc @yuhan
n

Noah K

01/08/2021, 6:22 PM
I was actually poking around for global/pipeline webhooks the other day too 🙂
Also pipeline level timeouts? Could implement it on our control end but it's weird to have pipelines in a "running" state forever
d

dwall

01/08/2021, 6:24 PM
would love these. We use start/end solids right now to ping the cronitor API. Would love to replace these with hooks instead!
👍 1
y

yuhan

01/08/2021, 6:24 PM
You can set up a hook that is across the pipeline https://docs.dagster.io/examples/hooks#apply-a-hook-on-a-pipeline
d

dwall

01/08/2021, 6:25 PM
@yuhan ah, I was thinking more on-start/on-end hooks
y

yuhan

01/08/2021, 6:25 PM
But we currently don’t have pipeline level hooks - all hooks are at step level. A hook on a pipeline means it is all every single step in the pipeline
Ah we don’t have that built yet re: on-start/end
n

Noah K

01/08/2021, 6:27 PM
Use case for me is integrating with a "control" application. You can start a bunch of jobs via graphql, but then it has to scrape them every X seconds to tell if they succeeded or crashed, and implement its own "this has been running too long, something probably went wrong" timeout
a

alex

01/08/2021, 6:27 PM
not yet is the answer on pipeline level timeouts as well - the
daemon
coming in
0.10.0
gives us the place to make that happen cc @johann
👍 3
y

yuhan

01/27/2021, 6:42 PM
FYI the pipeline level hook feature request is being tracked here https://github.com/dagster-io/dagster/issues/3613
👍 1
j

Josh Karlin

01/28/2021, 12:32 PM
thanks!