I have two pipelines, A and B. If I want B to run upon successful run of A, would using sensors be the right way to do it? I’m thinking pipeline A would have a solid to write a file to S3 to indicate it was done, and then a sensor would pick that up and runs pipeline B. Please let me know if there’s a better way.

Hi @Makoto! Yes, that’s a good way to set up that inter-pipeline dependency. We have asset sensors to check for materializations of certain external assets, which might be a particularly good fit for this use case. We’re also exploring ways of surfacing these inter-pipeline dependencies through assets in Dagit, so stay tuned for more updates on this front.

Thanks @prha!

Hi @prha, I want to use your approach but in my case, I have several pipelines (A, B, C, ...) to execute a pipeline Z ( It would consume the materializations of pipelines A, B, C, so on ). The pipelines A, B, C,... will have different schedulers at times/days. The pipeline Z only should be trigger if all the dependencies were success. Do you have something in mind to develop this? you said the pipelines can listening AssetKey, could the pipelines listen for more than one AssetKey and wait to get all the AssetKey needed?

@chrispc You can certainly have a sensor listening for more than one asset key, but you would have to write the logic that queries for the event records manually instead of using the

@asset_sensor

decorator. You could then update the cursor if and only if all of the A, B, C pipelines have emitted their corresponding materializations. I believe there’s some example code in the documentation for this: https://docs.dagster.io/concepts/partitions-schedules-sensors/sensors#multi-asset-sensors

thank you so much @prha

@prha I’d like to use the new

@run_status_sensor

decorator to kick off a pipeline B upon successful completion of a pipeline A. Is that possible? I don’t see how yielding a RunRequest would work in this case, since the

@run_status_sensor

decorator does not take a

pipeline_name

parameter to target for requested runs.