Hey folks! I have some old AWS Glue Jobs that I would like to integrate in a more modern pipeline. What I'd like to do in short is something like: Old jobs -> wait for successful completion -> trigger downstream stuff (other jobs, dbt runs etc...). I went through the docs and the getting started guide but I couldn't understand how Dagster can be notified about the job completion: do you know if it's possible to do it without polling ?

Hi Federio - one possibility would be to use the graphql api to launch the downstream stuff? https://docs.dagster.io/concepts/webserver/graphql#graphql-api That's the only non-polling option that we have that i'm aware of

Ah interesting! So as far as I understand, I would have to install the dagster graphQL python client alongside my glue job, and on successful execution, use it to trigger the rest of the pipelines, did I get it right?

The Graphql Python client is one way to trigger dagster graphql jobs, yeah - you can also use a standard python graphql client library like gql, it's just a bit more work to set up the queries

Got it, thanks a lot for the detailed explanation!