Discussion Topic: Versioning of Dags/Pipelines and display in dagit.
Hi I wanted to bring up discussion on Versioning of dags/pipelines. This is not talking about git versioning but rather treating a single pipeline [dag] as versioned artifact. Displaying the version in the UI so the pipeline executor can visually inspect what’s version is running
Deployment Modes:
1. Kubernetes User Deployments: One pipeline [dags] per docker image (deployment)
2. Kubernetes User Deployments: Many pipelines [dags] per docker image (deployment)
When working with docker images, the version is managed and deployed, but here the versioning stays limited to CI/CD and not exposed to end consumers/analysts. There is still simplicity due to 1:1 mapping between docker image release and pipeline release, but when working with multiple dags/pipelines within single docker image [user-deployment], the version traceability is mostly lost without any rigid tracking process
Is there a conventional way of displaying what dag / job version is running/deployed currently. Can we tag Job definition and display tags for the job on the UI ?
dagster bot responded by community 1