19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Channels
announcements
dagster-airbyte
dagster-airflow
dagster-bigquery
dagster-cloud
dagster-cube
dagster-dask
dagster-dbt
dagster-de
dagster-ecs
dagster-feedback
dagster-kubernetes
dagster-machine-learning
dagster-noteable
dagster-releases
dagster-serverless
dagster-showcase
dagster-snowflake
dagster-spatial
dagster-support
dagster-wandb
dagstereo
data-platform-design
events
faq-read-me-before-posting
gigs-freelance
github-discussions
introductions
jobs
random
tools
豆瓣酱帮
Title
a
Aaron
09/22/2021, 1:53 PMhello — is it possible to create a pipeline/graph that relies on the completion of a pipeline that is external to the former’s workspace?
y
yuhan
09/22/2021, 2:16 PMHi Aaron, would these two pipelines belong to the same dagster instance?
If they share the same instance, I think you can do so by creating a sensor that listens to either:
- run storage (via
context.instance.get_run_recordscontext.instance.get_event_records👍 1
If we are talking about across instances, you may still be able to do it using sensor, but likely need to introduce an intermediate external state. for example, the former could persist a complete marker in s3, and then you can set up a sensor in the latter’s workspace to listen to that s3 bucket.
❤️ 1