https://dagster.io/ logo
#ask-community
Title
# ask-community
h

Hebo Yang

08/17/2022, 6:46 PM
HI team, I am looking to build inter-job dependencies and trigger downstream jobs when an upstream job succeeds. It seems that `run_status_sensor`() is the way to go. I see that it support monitored_jobs and monitored_pipelines. For monitored_jobs, could I reference a job by its string name instead of the job object?
🤖 1
s

sandy

08/17/2022, 8:10 PM
This isn't currently possible, but we might be able to make it possible soon. Out of curiosity, what's the reason you'd prefer to use a string than a job object? Here's a related request that we got recently: https://github.com/dagster-io/dagster/issues/9394
e

Eegan K

08/17/2022, 11:19 PM
there are a few other ways to do this, but nothing exactly like what you're saying. For example, it's possible to trigger a job using a sensor if you create a new object in a database. Dagster also has an “asset sensor” which does something similar. Main limitation is that it does not trigger a job for each new asset materialization, just the latest one (although there is an open MR to fix that here: https://github.com/dagster-io/dagster/issues/5699)
h

Hebo Yang

08/18/2022, 6:20 PM
Thanks @sandy. I see the
run_status_sensor()
monitored_jobs param could take in an
UnresolvedAssetJobDefinition
. I am wondering if I could use
UnresolvedAssetJobDefinition
with an asset job name (or something?) to reference another job? (Context: we dynamically generate jobs based on yamls defined by our users. So when we are generating each job, we don’t have the complete references to all jobs.)
Hi @Eegan K, thanks for the pointer. If i set the limit to None and get back all records after prev cursor, is there a way for us to figure out which partition the asset materialization event is? Do I have to query with run_id for each event?
e

Eegan K

08/18/2022, 6:29 PM
To be completely honest I’m not sure. In my use case the cursor wasn’t useful, so I didn't look into it.
h

Hebo Yang

08/18/2022, 6:30 PM
I see. Thanks
s

sandy

08/18/2022, 9:26 PM
@Hebo Yang it's possible that your
UnresolvedAssetJobDefinition
scheme would work, but you'd need to try it out to be sure. when this change is merged, we should have a more supported way of doing this: https://github.com/dagster-io/dagster/pull/9405
h

Hebo Yang

08/18/2022, 9:27 PM
Got it. Thanks Sandy!
2 Views