The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Also, as a K8s/Helm deployer, I’m starting to be curious the best method to run the `dagster-airbyte apply` on the reconciler in each environment

My initial thought is Gitlab runner for CI/CD, but I assume the dagster-airbyte apply CLI needs to be applied in the `dagster-user-deployment`  K8s pod?

The CLI needs to be triggered in an environment in which it (1) has access to the Python code and (2) has access to your Airbyte instance - if your instance is visible to the internet, then a CI/CD process makes sense

If you’re running on Kubernetes and don’t have the instance accessible in your CI process, one option when developing locally is to use `kubectl port-forward …` and then run the CLI locally, pointed at the port-forwarded address

Thanks Ben, I’m fairly certain we can get the CI job running in the environment with the service accessible.  