19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Hi team, I have a pipeline A that does some pre-co...
# ask-communityh
Hebo Yang
08/27/2021, 9:37 PMHi team, I have a pipeline A that does some pre-computation and then waits for an asset from pipeline B before proceeding. Is there a way that I could create a solid in pipeline A to listen to asset materialization events from other pipelines please?
Hebo Yang
08/27/2021, 9:41 PMOne approach might be to combine A and B into a single pipeline. However, this will ends in a large pipeline which doesn’t correlate to business logic and also create problems for sensors. Another approach would be to break down pipeline A but again..it’s breaking down a group of things that logically belongs to the same pipeline.
m
max
08/27/2021, 9:52 PMHi Hebo, have you looked at the GraphQL API at all?
h
Hebo Yang
08/27/2021, 10:01 PMNot yet. Should I use graphql api to query for specific materialization event?
a
Arun Kumar
08/27/2021, 10:29 PMIs n't this a use-case for sensors? Can't the sensor for pipeline A listen for the assent events produced by the pipeline B before triggering the request?
m
max
08/27/2021, 10:46 PMyep @Arun Kumar -- that will certainly work if we want to trigger the whole pipeline (vs. having a single solid in the pipeline wait)
h
Hebo Yang
08/27/2021, 10:46 PMI already have a sensor for pipeline A so that it could start to do the pre-computation.
Hebo Yang
08/27/2021, 10:49 PMBut need then to wait for some other dependency. This way, the job finishes up earlier.
a
Arun Kumar
08/27/2021, 11:45 PMI see. May be I am missing something fundamental about your use case. Why would you not want to break it into different pipelines? Apparently it feels like it should be a different pipeline as it has a different dependency. Also, since we are running a pipeline in a separate k8s pod, if the assets from Pipeline B took a long time to appear, pipeline A will be holding the pod unnecessarily.
m
max
08/27/2021, 11:49 PMit might make sense to have three pipelines instead -- so that pipeline A does pre-computation, pipeline B does its work, and pipeline C waits on both of them
👍 1
max
08/27/2021, 11:50 PMotherwise yeah, you will essentially end up reproducing sensor logic in the solid that waits
h
Hebo Yang
08/30/2021, 6:45 PMThanks Max and Arun! Yeah, I ended in creating 3 pipelines and 3 sensors.