I wonder if I missed something in the docs... but I didn't find anything about how to coordinate the launch of multiple pipelines... e.g. I've Pipelines A and B and want to trigger Pipeline B right after Pipeline A is finished. How do you do this?

I believe the idea is that Pipeline A should have some sort of external asset which is created or updated (state changes) and then you use

Sensors

to trigger Pipeline B. Doc links below but the blog post has a better example of Team A creating an asset which Team B relies upon prior to triggering a pipeline https://docs.dagster.io/overview/schedules-sensors/sensors

So there is no direct mechanism for pipelines that are in the same repository anyways? e.g. calling pipelines from a meta-pipeline?

Not that I'm aware of

We're considering doing this by communicating with Dagster via GraphQL from inside a solid, perhaps the final solid in a pipeline, and "manually" kicking off another pipeline run. Can't attest to a successful experiment yet, but seems plausible 🙂

Sensors are one recommended way to do this today as Jason says - if you want to compose two pipelines in the same repository into a single larger pipeline, this upcoming feature will allow you to do that directly: https://github.com/dagster-io/dagster/discussions/2902 .

Thats great 🙂 What is the currently expected release date for 0.11?

@Klaus Stadler I believe we are aiming for some time in March. Out of curiosity, why separate these into two pipelines? Is it mainly for organizational purposes? Do you sometimes want to run one of them without the other one.

We certainly have this use case that you mention @sandy (sometimes wanting to run one without the other, also, wanting it to halt at a particular point based on some kind of external or human driven condition - at other times permanently stop at a certain point)

Thanks @esztermarton - that makes sense

Hi @sandy, our case is pretty much the same

likewise