hello — is it possible to create a pipeline/graph that relies on the completion of a pipeline that is external to the former’s workspace?
09/22/2021, 2:16 PM
Hi Aaron, would these two pipelines belong to the same dagster instance?
If they share the same instance, I think you can do so by creating a sensor that listens to either:
- run storage (via
- event storage (via
If we are talking about across instances, you may still be able to do it using sensor, but likely need to introduce an intermediate external state. for example, the former could persist a complete marker in s3, and then you can set up a sensor in the latter’s workspace to listen to that s3 bucket.