Hello Dagster community! I’ve been following Dagsters progress for a little more than a year and I finally feel like it has reached the level of maturity that we need to replace our Airflow setup! Let’s be honest though its had some features I’ve been salivating over way way before now haha. I’ve been going over the architecture and and it feels pretty straightforward for me to setup in our EKS cluster with the exception that we only use Terraform in our org and are not able to use helm. That being said, I feel pretty confidence I can easily port helm charts to Terraform no problem but I’m wondering if there are any major benefits I might be missing out on by not using helm? I haven’t looked into it to deeply yet but one thing I was thinking I might miss out on is the user repository deployments? Anyway, just curious is anyone has deployed their Dagster infra in Kuberentes without using helm and has any thoughts/feeling on it they would like to share!

You’ve likely already seen this but there is a helm provider for terraform https://registry.terraform.io/providers/hashicorp/helm/latest/docs

I have we just aren’t able to use it in our org for whatever reason ha 😅

@Kyle Hamlin I have installed Dagster on our GKE cluster using exactly that provider, what error are you getting? With that said, you can probably render the helm chart and convert it to terraform by yourself, but it's going to be a pain to maintain / upgrade.

So you’ll be looking to set things up with https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs?

Yeah it's a company policy

You can render the helm chart with the values you want and then pipe every resource through https://github.com/jrhouston/tfk8s

But possibly worth changing or getting an exception for in this case

Happy to answer any questions about the k8s deployment as they come up. I don’t think that you’ll necessarily be missing features, but as you point out things like user code deployments would take a bit of work to replicate. You can probably start simpler, running things within a single pod that matches how you’d run locally

Yep I've done that in the past when porting other helm charts haha

If you are not aware of it, it is possible to import airflow workflows in dagster: https://docs.dagster.io/integrations/airflow#ingesting-dags-from-airflow That's always a good starting point 🙂

Yeah I did see this and I gave it a shot yesterday but I think that functionality might be a bit out of date with the airflow code base because I was getting import errors for airflows BaseOperator. Happy to share the details if you're interested.

A stacktrace/error message would be great, maybe in #dagster-airflow

Cool will do!