Hello. I have job A that can run large numbers of ...
# ask-community
Hello. I have job A that can run large numbers of runs in parallel, each run creates an asset. I have an asset sensor running every second - the idea is each asset created by job A triggers a run of job B. Unfortunately the asset sensor misses some materialisation (when two job A runs finish at the same time). How can I make it so that:
Copy code
number of A runs == number of B runs
🤖 1
For instance, is it possible to yield a
from inside an op?
It seems like this is not possible, I will refactor my pipeline to combine both jobs into one.
Hi Archie, Are you still having problems with this? If so, by:
each asset created by job A
Do you mean each materialization of a single asset?
Maybe these two can help: https://docs.dagster.io/concepts/partitions-schedules-sensors/asset-sensors#asset-reconciliation-sensor https://docs.dagster.io/concepts/partitions-schedules-sensors/asset-sensors#triggering-runs-upon-materializations The second link is specifically for multi-assets but you can probably re-use that code to ensure that unconsumed materializations will trigger jobs
Thanks for the help all! But in the end it made sense to use a single job and multiple graphs for my architecture 🙂