Hey folks! Is it possible to version your pipelines? Also, is there a way to integrate with GitHub so that pipeline code gets updated when new commits are pushed upstream?

Can you define exactly what you mean by versioning? We persist serialized representations of our graphs so one can, for example, view the shape of a pipeline attached to a run at the time it was run, rather than the current shape.

I mean in the way that Prefect versions pipelines when Flows are updated and registered again

I’m not familiar with the way that Prefect expresses versioning. How would you expect it to work in this case?

It lets you track that the pipeline has changed and a particular run was run on say,

x

version of the pipeline

Yeah we have all that information. We don’t surface it too aggressively in our UI right now other than that runs backlink to the correct version of the pipeline. It’s definitely something we could surface easily. When do you think you would find it the most useful?

I'm looking for a way to integrate GitHub such that when new code is pushed to a pipeline's repo, it gets updated in Dagster and then I can see which runs were on newer or older versions of the pipeline This brings me back to my second question, is it possible to integrate GitHub in this manner?